.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "checks_gallery/tabular/data_integrity/plot_data_duplicates.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        Click :ref:`here <sphx_glr_download_checks_gallery_tabular_data_integrity_plot_data_duplicates.py>`
        to download the full example code

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_checks_gallery_tabular_data_integrity_plot_data_duplicates.py:


.. _plot_tabular_data_duplicates:

Data Duplicates
***************

This notebooks provides an overview for using and understanding the data duplicates check:

**Structure:**

* `Why data duplicates? <#why-data-duplicates>`__
* `Load Data <#load-data>`__
* `Run the Check <#run-the-check>`__
* `Define a Condition <#define-a-condition>`__

.. GENERATED FROM PYTHON SOURCE LINES 19-26

.. code-block:: default


    from datetime import datetime

    import pandas as pd

    from deepchecks.tabular.datasets.classification.phishing import load_data








.. GENERATED FROM PYTHON SOURCE LINES 27-38

Why data duplicates?
====================
The ``DataDuplicates`` check finds multiple instances of identical samples in the
Dataset. Duplicate samples increase the weight the model gives to those samples.
If these duplicates are there intentionally (e.g. as a result of intentional
oversampling, or due to the dataset's nature it has identical-looking samples)
this may be valid, however if this is an hidden issue we're not expecting to occur,
it may be an indicator for a problem in the data pipeline that requires attention.

Load Data
=========

.. GENERATED FROM PYTHON SOURCE LINES 38-43

.. code-block:: default



    phishing_dataset = load_data(as_train_test=False, data_format='DataFrame')
    phishing_dataset






.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>target</th>
          <th>month</th>
          <th>scrape_date</th>
          <th>ext</th>
          <th>urlLength</th>
          <th>...</th>
          <th>specialChars</th>
          <th>scriptLength</th>
          <th>sbr</th>
          <th>bscr</th>
          <th>sscr</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>0</th>
          <td>0</td>
          <td>1</td>
          <td>2019-01-01</td>
          <td>net</td>
          <td>102</td>
          <td>...</td>
          <td>9419</td>
          <td>23919</td>
          <td>0.736286</td>
          <td>0.289940</td>
          <td>2.539442</td>
        </tr>
        <tr>
          <th>1</th>
          <td>0</td>
          <td>1</td>
          <td>2019-01-01</td>
          <td>country</td>
          <td>154</td>
          <td>...</td>
          <td>2735</td>
          <td>794</td>
          <td>0.049015</td>
          <td>0.168838</td>
          <td>0.290311</td>
        </tr>
        <tr>
          <th>2</th>
          <td>0</td>
          <td>1</td>
          <td>2019-01-01</td>
          <td>net</td>
          <td>171</td>
          <td>...</td>
          <td>27798</td>
          <td>83817</td>
          <td>0.811049</td>
          <td>0.268985</td>
          <td>2.412174</td>
        </tr>
        <tr>
          <th>3</th>
          <td>0</td>
          <td>1</td>
          <td>2019-01-01</td>
          <td>com</td>
          <td>94</td>
          <td>...</td>
          <td>9087</td>
          <td>19427</td>
          <td>0.569824</td>
          <td>0.266536</td>
          <td>2.137889</td>
        </tr>
        <tr>
          <th>4</th>
          <td>0</td>
          <td>1</td>
          <td>2019-01-01</td>
          <td>other</td>
          <td>95</td>
          <td>...</td>
          <td>39</td>
          <td>0</td>
          <td>0.000000</td>
          <td>0.193069</td>
          <td>0.000000</td>
        </tr>
        <tr>
          <th>...</th>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
        </tr>
        <tr>
          <th>11345</th>
          <td>0</td>
          <td>1</td>
          <td>2020-01-15</td>
          <td>country</td>
          <td>89</td>
          <td>...</td>
          <td>971</td>
          <td>1866</td>
          <td>0.625302</td>
          <td>0.213266</td>
          <td>2.932029</td>
        </tr>
        <tr>
          <th>11346</th>
          <td>0</td>
          <td>1</td>
          <td>2020-01-15</td>
          <td>other</td>
          <td>107</td>
          <td>...</td>
          <td>3185</td>
          <td>4228</td>
          <td>0.291069</td>
          <td>0.214348</td>
          <td>1.357928</td>
        </tr>
        <tr>
          <th>11347</th>
          <td>0</td>
          <td>1</td>
          <td>2020-01-15</td>
          <td>com</td>
          <td>112</td>
          <td>...</td>
          <td>0</td>
          <td>0</td>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.000000</td>
        </tr>
        <tr>
          <th>11348</th>
          <td>0</td>
          <td>1</td>
          <td>2020-01-15</td>
          <td>html</td>
          <td>111</td>
          <td>...</td>
          <td>0</td>
          <td>0</td>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.000000</td>
        </tr>
        <tr>
          <th>11349</th>
          <td>0</td>
          <td>1</td>
          <td>2020-01-15</td>
          <td>html</td>
          <td>97</td>
          <td>...</td>
          <td>25</td>
          <td>0</td>
          <td>0.000000</td>
          <td>0.167785</td>
          <td>0.000000</td>
        </tr>
      </tbody>
    </table>
    <p>11350 rows × 25 columns</p>
    </div>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 44-46

Run the Check
=============

.. GENERATED FROM PYTHON SOURCE LINES 46-58

.. code-block:: default


    from deepchecks.tabular.checks import DataDuplicates

    DataDuplicates().run(phishing_dataset)

    # With Check Parameters
    # ---------------------
    # ``DataDuplicates`` check can also use a specific subset of columns (or alternatively
    # use all columns except specific ignore_columns to check duplication):

    DataDuplicates(columns=["entropy", "numParams"]).run(phishing_dataset)






.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <!--
        ~ ----------------------------------------------------------------------------
        ~ Copyright (C) 2021-2022 Deepchecks (https://www.deepchecks.com)
        ~
        ~ This file is part of Deepchecks.
        ~ Deepchecks is distributed under the terms of the GNU Affero General
        ~ Public License (version 3 or later).
        ~ You should have received a copy of the GNU Affero General Public License
        ~ along with Deepchecks.  If not, see <http://www.gnu.org/licenses/>.
        ~ ----------------------------------------------------------------------------
        ~
    -->
    <!DOCTYPE html>
    <html lang="en">
    <head>
        <meta charset="UTF-8">
        <title>Data Duplicates</title>
        <style>
            body {
                font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Helvetica, Arial, sans-serif, 'Apple Color Emoji', 'Segoe UI Emoji', 'Segoe UI Symbol';
                font-size: 1rem;
                line-height: 1.5;
                color: #212529;
                text-align: left;
                max-width: 1200px;
                margin: auto;
                background-color: white;
            }

            div.nbinput.container div.prompt *,
            div.nboutput.container div.prompt *,
            div.nbinput.container div.input_area pre,
            div.nboutput.container div.output_area pre,
            div.nbinput.container div.input_area .highlight,
            div.nboutput.container div.output_area .highlight {
                border: none;
                padding: 0;
                margin: 0;
                box-shadow: none;
            }

            div.nbinput.container > div[class*=highlight],
            div.nboutput.container > div[class*=highlight] {
                margin: 0;
            }

            div.nbinput.container div.prompt *,
            div.nboutput.container div.prompt * {
                background: none;
            }

            div.nboutput.container div.output_area .highlight,
            div.nboutput.container div.output_area pre {
                background: unset;
            }

            div.nboutput.container div.output_area div.highlight {
                color: unset;  /* override Pygments text color */
            }

            /* avoid gaps between output lines */
            div.nboutput.container div[class*=highlight] pre {
                line-height: normal;
            }

            /* input/output containers */
            div.nbinput.container,
            div.nboutput.container {
                display: -webkit-flex;
                display: flex;
                align-items: flex-start;
                margin: 0;
                width: 100%;
            }
            @media (max-width: 540px) {
                div.nbinput.container,
                div.nboutput.container {
                    flex-direction: column;
                }
            }

            /* input container */
            div.nbinput.container {
                padding-top: 5px;
            }

            /* last container */
            div.nblast.container {
                padding-bottom: 5px;
            }

            /* input prompt */
            div.nbinput.container div.prompt pre {
                color: #307FC1;
            }

            /* output prompt */
            div.nboutput.container div.prompt pre {
                color: #BF5B3D;
            }

            /* all prompts */
            div.nbinput.container div.prompt,
            div.nboutput.container div.prompt {
                width: 4.5ex;
                padding-top: 5px;
                position: relative;
                user-select: none;
            }

            div.nbinput.container div.prompt > div,
            div.nboutput.container div.prompt > div {
                position: absolute;
                right: 0;
                margin-right: 0.3ex;
            }

            @media (max-width: 540px) {
                div.nbinput.container div.prompt,
                div.nboutput.container div.prompt {
                    width: unset;
                    text-align: left;
                    padding: 0.4em;
                }
                div.nboutput.container div.prompt.empty {
                    padding: 0;
                }

                div.nbinput.container div.prompt > div,
                div.nboutput.container div.prompt > div {
                    position: unset;
                }
            }

            /* disable scrollbars on prompts */
            div.nbinput.container div.prompt pre,
            div.nboutput.container div.prompt pre {
                overflow: hidden;
            }

            /* input/output area */
            div.nbinput.container div.input_area,
            div.nboutput.container div.output_area {
                -webkit-flex: 1;
                flex: 1;
                overflow: auto;
            }
            @media (max-width: 540px) {
                div.nbinput.container div.input_area,
                div.nboutput.container div.output_area {
                    width: 100%;
                }
            }

            /* input area */
            div.nbinput.container div.input_area {
                border: 1px solid #e0e0e0;
                border-radius: 2px;
                /*background: #f5f5f5;*/
            }

            /* override MathJax center alignment in output cells */
            div.nboutput.container div[class*=MathJax] {
                text-align: left !important;
            }

            /* override sphinx.ext.imgmath center alignment in output cells */
            div.nboutput.container div.math p {
                text-align: left;
            }

            /* standard error */
            div.nboutput.container div.output_area.stderr {
                background: #fdd;
            }

            /* ANSI colors */
            .ansi-black-fg { color: #3E424D; }
            .ansi-black-bg { background-color: #3E424D; }
            .ansi-black-intense-fg { color: #282C36; }
            .ansi-black-intense-bg { background-color: #282C36; }
            .ansi-red-fg { color: #E75C58; }
            .ansi-red-bg { background-color: #E75C58; }
            .ansi-red-intense-fg { color: #B22B31; }
            .ansi-red-intense-bg { background-color: #B22B31; }
            .ansi-green-fg { color: #00A250; }
            .ansi-green-bg { background-color: #00A250; }
            .ansi-green-intense-fg { color: #007427; }
            .ansi-green-intense-bg { background-color: #007427; }
            .ansi-yellow-fg { color: #DDB62B; }
            .ansi-yellow-bg { background-color: #DDB62B; }
            .ansi-yellow-intense-fg { color: #B27D12; }
            .ansi-yellow-intense-bg { background-color: #B27D12; }
            .ansi-blue-fg { color: #208FFB; }
            .ansi-blue-bg { background-color: #208FFB; }
            .ansi-blue-intense-fg { color: #0065CA; }
            .ansi-blue-intense-bg { background-color: #0065CA; }
            .ansi-magenta-fg { color: #D160C4; }
            .ansi-magenta-bg { background-color: #D160C4; }
            .ansi-magenta-intense-fg { color: #A03196; }
            .ansi-magenta-intense-bg { background-color: #A03196; }
            .ansi-cyan-fg { color: #60C6C8; }
            .ansi-cyan-bg { background-color: #60C6C8; }
            .ansi-cyan-intense-fg { color: #258F8F; }
            .ansi-cyan-intense-bg { background-color: #258F8F; }
            .ansi-white-fg { color: #C5C1B4; }
            .ansi-white-bg { background-color: #C5C1B4; }
            .ansi-white-intense-fg { color: #A1A6B2; }
            .ansi-white-intense-bg { background-color: #A1A6B2; }

            .ansi-default-inverse-fg { color: #FFFFFF; }
            .ansi-default-inverse-bg { background-color: #000000; }

            .ansi-bold { font-weight: bold; }
            .ansi-underline { text-decoration: underline; }


            div.nbinput.container div.input_area div[class*=highlight] > pre,
            div.nboutput.container div.output_area div[class*=highlight] > pre,
            div.nboutput.container div.output_area div[class*=highlight].math,
            div.nboutput.container div.output_area.rendered_html,
            div.nboutput.container div.output_area > div.output_javascript,
            div.nboutput.container div.output_area:not(.rendered_html) > img{
                padding: 5px;
                margin: 0;
            }

            /* fix copybtn overflow problem in chromium (needed for 'sphinx_copybutton') */
            div.nbinput.container div.input_area > div[class^='highlight'],
            div.nboutput.container div.output_area > div[class^='highlight']{
                overflow-y: hidden;
            }

            /* hide copybtn icon on prompts (needed for 'sphinx_copybutton') */
            .prompt .copybtn {
                display: none;
            }

            /* Some additional styling taken form the Jupyter notebook CSS */
            div.rendered_html table {
            border: none;
            border-collapse: collapse;
            border-spacing: 0;
            color: black;
            font-size: 12px;
            table-layout: fixed;
            }
            div.rendered_html thead {
            border-bottom: 1px solid black;
            vertical-align: bottom;
            }
            div.rendered_html tr,
            div.rendered_html th,
            div.rendered_html td {
            text-align: right;
            vertical-align: middle;
            padding: 0.5em 0.5em;
            line-height: normal;
            white-space: normal;
            max-width: none;
            border: none;
            }
            div.rendered_html th {
            font-weight: bold;
            }
            div.rendered_html tbody tr:nth-child(odd) {
            background: #f5f5f5;
            }
            div.rendered_html tbody tr:hover {
            background: rgba(66, 165, 245, 0.2);
            }
        </style>
    </head>
    <body>
        <script src="https://unpkg.com/@jupyter-widgets/html-manager@^0.20.1/dist/embed.js" crossorigin="anonymous"></script>
    

    <script type="application/vnd.jupyter.widget-state+json">
    {"version_major": 2, "version_minor": 0, "state": {"20d32c29a1e44ec1834276b69953d560": {"model_name": "VBoxModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": ["rendered_html", "jp-RenderedHTMLCommon", "jp-RenderedHTML", "jp-OutputArea-output"], "children": ["IPY_MODEL_b302c30f689f43418c7badcbf8c9b845", "IPY_MODEL_efee54b0e38944318f65f474ed85d442", "IPY_MODEL_ec2a609e1030461a8714426b6c443b07", "IPY_MODEL_590174d43e6140b094f4a3b83229e87d"], "layout": "IPY_MODEL_2d2c36f1458b4c3bbe35cfa17eb99222"}}, "b302c30f689f43418c7badcbf8c9b845": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_6da9d5ba401d4e648dc4a3db272ec777", "style": "IPY_MODEL_a7a7f05eaca64bc3aee8dfcb117725d4", "value": "<h4><b>Data Duplicates</b></h4>"}}, "6da9d5ba401d4e648dc4a3db272ec777": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "a7a7f05eaca64bc3aee8dfcb117725d4": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "efee54b0e38944318f65f474ed85d442": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_f1fb7e04c8f84109acbe2dc04fe4560c", "style": "IPY_MODEL_4b3fd57c51d4415f9a5cbf8d26c22735", "value": "<p>Checks for duplicate samples in the dataset. <a href=\"https://docs.deepchecks.com/0.8/checks_gallery/tabular/data_integrity/plot_data_duplicates.html?utm_source=display_output&utm_medium=referral&utm_campaign=check_link\" target=\"_blank\">Read More...</a></p>"}}, "f1fb7e04c8f84109acbe2dc04fe4560c": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "4b3fd57c51d4415f9a5cbf8d26c22735": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "ec2a609e1030461a8714426b6c443b07": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_02d30b9a70f94e5292e49494a273a00e", "style": "IPY_MODEL_596d30da7cae485fa0179e89bec10ce0"}}, "02d30b9a70f94e5292e49494a273a00e": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "596d30da7cae485fa0179e89bec10ce0": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "590174d43e6140b094f4a3b83229e87d": {"model_name": "VBoxModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "children": ["IPY_MODEL_136af0696cdc44ca87bd9851af59920e", "IPY_MODEL_53aa98c3361f438ca22bbee8f7578223", "IPY_MODEL_0a52e31a44b94440a9c04e1410fee84f", "IPY_MODEL_b384499cb8dc41f08edb8f6fd614b147"], "layout": "IPY_MODEL_bacd55e063a44dceabed3b074bb40b9d"}}, "136af0696cdc44ca87bd9851af59920e": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_ec870a26561343d7acd6739fe9e828c5", "style": "IPY_MODEL_c5d0e52bed494f21a40d616e05780b1f", "value": "<h5><b>Additional Outputs</b></h5>"}}, "ec870a26561343d7acd6739fe9e828c5": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "c5d0e52bed494f21a40d616e05780b1f": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "53aa98c3361f438ca22bbee8f7578223": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_95ffb4aa60644e5eaaf6a24b1b3ca9f7", "style": "IPY_MODEL_fd2aa6d3dc8b44fd859f4b0d62682059", "value": "<div>4.11% of data samples are duplicates. </div>"}}, "95ffb4aa60644e5eaaf6a24b1b3ca9f7": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "fd2aa6d3dc8b44fd859f4b0d62682059": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "0a52e31a44b94440a9c04e1410fee84f": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_bde0a63dfc4b415a9e65e3b9e37998da", "style": "IPY_MODEL_7b0603357b56456d97134f1372306244", "value": "<div>Each row in the table shows an example of duplicate data and the number of times it appears.</div>"}}, "bde0a63dfc4b415a9e65e3b9e37998da": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "7b0603357b56456d97134f1372306244": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "b384499cb8dc41f08edb8f6fd614b147": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_7602d2ee41ad4a179bf10c8a07beef49", "style": "IPY_MODEL_17de186748dc44628526a869cbeb3e64", "value": "<style type=\"text/css\">\n#T_4cb37_ table {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_4cb37_ thead {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_4cb37_ tbody {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_4cb37_ th {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_4cb37_ td {\n  text-align: left;\n  white-space: pre-wrap;\n}\n</style>\n<table id=\"T_4cb37_\">\n  <thead>\n    <tr>\n      <th class=\"blank\" >&nbsp;</th>\n      <th class=\"blank level0\" >&nbsp;</th>\n      <th class=\"col_heading level0 col0\" >entropy</th>\n      <th class=\"col_heading level0 col1\" >numParams</th>\n    </tr>\n    <tr>\n      <th class=\"index_name level0\" >Instances</th>\n      <th class=\"index_name level1\" >Number of Duplicates</th>\n      <th class=\"blank col0\" >&nbsp;</th>\n      <th class=\"blank col1\" >&nbsp;</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th id=\"T_4cb37_level0_row0\" class=\"row_heading level0 row0\" >82, 974, 1557, 2150, 2360, 3528, 6560, 7...</th>\n      <th id=\"T_4cb37_level1_row0\" class=\"row_heading level1 row0\" >13</th>\n      <td id=\"T_4cb37_row0_col0\" class=\"data row0 col0\" >-4.31</td>\n      <td id=\"T_4cb37_row0_col1\" class=\"data row0 col1\" >0</td>\n    </tr>\n    <tr>\n      <th id=\"T_4cb37_level0_row1\" class=\"row_heading level0 row1\" >1641, 1729, 2213, 2234, 4412, 4638, 6328...</th>\n      <th id=\"T_4cb37_level1_row1\" class=\"row_heading level1 row1\" >8</th>\n      <td id=\"T_4cb37_row1_col0\" class=\"data row1 col0\" >-4.57</td>\n      <td id=\"T_4cb37_row1_col1\" class=\"data row1 col1\" >4</td>\n    </tr>\n    <tr>\n      <th id=\"T_4cb37_level0_row2\" class=\"row_heading level0 row2\" >2719, 4634, 6504, 6774, 6783, 7528, 9592...</th>\n      <th id=\"T_4cb37_level1_row2\" class=\"row_heading level1 row2\" >8</th>\n      <td id=\"T_4cb37_row2_col0\" class=\"data row2 col0\" >-4.49</td>\n      <td id=\"T_4cb37_row2_col1\" class=\"data row2 col1\" >8</td>\n    </tr>\n    <tr>\n      <th id=\"T_4cb37_level0_row3\" class=\"row_heading level0 row3\" >929, 2499, 4047, 7989, 8391, 9348, 9932,...</th>\n      <th id=\"T_4cb37_level1_row3\" class=\"row_heading level1 row3\" >8</th>\n      <td id=\"T_4cb37_row3_col0\" class=\"data row3 col0\" >-4.25</td>\n      <td id=\"T_4cb37_row3_col1\" class=\"data row3 col1\" >0</td>\n    </tr>\n    <tr>\n      <th id=\"T_4cb37_level0_row4\" class=\"row_heading level0 row4\" >1020, 1670, 1802, 2984, 6666, 9138, 1092...</th>\n      <th id=\"T_4cb37_level1_row4\" class=\"row_heading level1 row4\" >7</th>\n      <td id=\"T_4cb37_row4_col0\" class=\"data row4 col0\" >-4.65</td>\n      <td id=\"T_4cb37_row4_col1\" class=\"data row4 col1\" >5</td>\n    </tr>\n  </tbody>\n</table>\n"}}, "7602d2ee41ad4a179bf10c8a07beef49": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "17de186748dc44628526a869cbeb3e64": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "bacd55e063a44dceabed3b074bb40b9d": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "2d2c36f1458b4c3bbe35cfa17eb99222": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}}}
    </script>
    <script type="application/vnd.jupyter.widget-view+json">
    {"version_major": 2, "version_minor": 0, "model_id": "20d32c29a1e44ec1834276b69953d560"}
    </script>

    </body>
    </html>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 59-62

.. code-block:: default


    DataDuplicates(ignore_columns=["scrape_date"], n_to_show=10).run(phishing_dataset)






.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <!--
        ~ ----------------------------------------------------------------------------
        ~ Copyright (C) 2021-2022 Deepchecks (https://www.deepchecks.com)
        ~
        ~ This file is part of Deepchecks.
        ~ Deepchecks is distributed under the terms of the GNU Affero General
        ~ Public License (version 3 or later).
        ~ You should have received a copy of the GNU Affero General Public License
        ~ along with Deepchecks.  If not, see <http://www.gnu.org/licenses/>.
        ~ ----------------------------------------------------------------------------
        ~
    -->
    <!DOCTYPE html>
    <html lang="en">
    <head>
        <meta charset="UTF-8">
        <title>Data Duplicates</title>
        <style>
            body {
                font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Helvetica, Arial, sans-serif, 'Apple Color Emoji', 'Segoe UI Emoji', 'Segoe UI Symbol';
                font-size: 1rem;
                line-height: 1.5;
                color: #212529;
                text-align: left;
                max-width: 1200px;
                margin: auto;
                background-color: white;
            }

            div.nbinput.container div.prompt *,
            div.nboutput.container div.prompt *,
            div.nbinput.container div.input_area pre,
            div.nboutput.container div.output_area pre,
            div.nbinput.container div.input_area .highlight,
            div.nboutput.container div.output_area .highlight {
                border: none;
                padding: 0;
                margin: 0;
                box-shadow: none;
            }

            div.nbinput.container > div[class*=highlight],
            div.nboutput.container > div[class*=highlight] {
                margin: 0;
            }

            div.nbinput.container div.prompt *,
            div.nboutput.container div.prompt * {
                background: none;
            }

            div.nboutput.container div.output_area .highlight,
            div.nboutput.container div.output_area pre {
                background: unset;
            }

            div.nboutput.container div.output_area div.highlight {
                color: unset;  /* override Pygments text color */
            }

            /* avoid gaps between output lines */
            div.nboutput.container div[class*=highlight] pre {
                line-height: normal;
            }

            /* input/output containers */
            div.nbinput.container,
            div.nboutput.container {
                display: -webkit-flex;
                display: flex;
                align-items: flex-start;
                margin: 0;
                width: 100%;
            }
            @media (max-width: 540px) {
                div.nbinput.container,
                div.nboutput.container {
                    flex-direction: column;
                }
            }

            /* input container */
            div.nbinput.container {
                padding-top: 5px;
            }

            /* last container */
            div.nblast.container {
                padding-bottom: 5px;
            }

            /* input prompt */
            div.nbinput.container div.prompt pre {
                color: #307FC1;
            }

            /* output prompt */
            div.nboutput.container div.prompt pre {
                color: #BF5B3D;
            }

            /* all prompts */
            div.nbinput.container div.prompt,
            div.nboutput.container div.prompt {
                width: 4.5ex;
                padding-top: 5px;
                position: relative;
                user-select: none;
            }

            div.nbinput.container div.prompt > div,
            div.nboutput.container div.prompt > div {
                position: absolute;
                right: 0;
                margin-right: 0.3ex;
            }

            @media (max-width: 540px) {
                div.nbinput.container div.prompt,
                div.nboutput.container div.prompt {
                    width: unset;
                    text-align: left;
                    padding: 0.4em;
                }
                div.nboutput.container div.prompt.empty {
                    padding: 0;
                }

                div.nbinput.container div.prompt > div,
                div.nboutput.container div.prompt > div {
                    position: unset;
                }
            }

            /* disable scrollbars on prompts */
            div.nbinput.container div.prompt pre,
            div.nboutput.container div.prompt pre {
                overflow: hidden;
            }

            /* input/output area */
            div.nbinput.container div.input_area,
            div.nboutput.container div.output_area {
                -webkit-flex: 1;
                flex: 1;
                overflow: auto;
            }
            @media (max-width: 540px) {
                div.nbinput.container div.input_area,
                div.nboutput.container div.output_area {
                    width: 100%;
                }
            }

            /* input area */
            div.nbinput.container div.input_area {
                border: 1px solid #e0e0e0;
                border-radius: 2px;
                /*background: #f5f5f5;*/
            }

            /* override MathJax center alignment in output cells */
            div.nboutput.container div[class*=MathJax] {
                text-align: left !important;
            }

            /* override sphinx.ext.imgmath center alignment in output cells */
            div.nboutput.container div.math p {
                text-align: left;
            }

            /* standard error */
            div.nboutput.container div.output_area.stderr {
                background: #fdd;
            }

            /* ANSI colors */
            .ansi-black-fg { color: #3E424D; }
            .ansi-black-bg { background-color: #3E424D; }
            .ansi-black-intense-fg { color: #282C36; }
            .ansi-black-intense-bg { background-color: #282C36; }
            .ansi-red-fg { color: #E75C58; }
            .ansi-red-bg { background-color: #E75C58; }
            .ansi-red-intense-fg { color: #B22B31; }
            .ansi-red-intense-bg { background-color: #B22B31; }
            .ansi-green-fg { color: #00A250; }
            .ansi-green-bg { background-color: #00A250; }
            .ansi-green-intense-fg { color: #007427; }
            .ansi-green-intense-bg { background-color: #007427; }
            .ansi-yellow-fg { color: #DDB62B; }
            .ansi-yellow-bg { background-color: #DDB62B; }
            .ansi-yellow-intense-fg { color: #B27D12; }
            .ansi-yellow-intense-bg { background-color: #B27D12; }
            .ansi-blue-fg { color: #208FFB; }
            .ansi-blue-bg { background-color: #208FFB; }
            .ansi-blue-intense-fg { color: #0065CA; }
            .ansi-blue-intense-bg { background-color: #0065CA; }
            .ansi-magenta-fg { color: #D160C4; }
            .ansi-magenta-bg { background-color: #D160C4; }
            .ansi-magenta-intense-fg { color: #A03196; }
            .ansi-magenta-intense-bg { background-color: #A03196; }
            .ansi-cyan-fg { color: #60C6C8; }
            .ansi-cyan-bg { background-color: #60C6C8; }
            .ansi-cyan-intense-fg { color: #258F8F; }
            .ansi-cyan-intense-bg { background-color: #258F8F; }
            .ansi-white-fg { color: #C5C1B4; }
            .ansi-white-bg { background-color: #C5C1B4; }
            .ansi-white-intense-fg { color: #A1A6B2; }
            .ansi-white-intense-bg { background-color: #A1A6B2; }

            .ansi-default-inverse-fg { color: #FFFFFF; }
            .ansi-default-inverse-bg { background-color: #000000; }

            .ansi-bold { font-weight: bold; }
            .ansi-underline { text-decoration: underline; }


            div.nbinput.container div.input_area div[class*=highlight] > pre,
            div.nboutput.container div.output_area div[class*=highlight] > pre,
            div.nboutput.container div.output_area div[class*=highlight].math,
            div.nboutput.container div.output_area.rendered_html,
            div.nboutput.container div.output_area > div.output_javascript,
            div.nboutput.container div.output_area:not(.rendered_html) > img{
                padding: 5px;
                margin: 0;
            }

            /* fix copybtn overflow problem in chromium (needed for 'sphinx_copybutton') */
            div.nbinput.container div.input_area > div[class^='highlight'],
            div.nboutput.container div.output_area > div[class^='highlight']{
                overflow-y: hidden;
            }

            /* hide copybtn icon on prompts (needed for 'sphinx_copybutton') */
            .prompt .copybtn {
                display: none;
            }

            /* Some additional styling taken form the Jupyter notebook CSS */
            div.rendered_html table {
            border: none;
            border-collapse: collapse;
            border-spacing: 0;
            color: black;
            font-size: 12px;
            table-layout: fixed;
            }
            div.rendered_html thead {
            border-bottom: 1px solid black;
            vertical-align: bottom;
            }
            div.rendered_html tr,
            div.rendered_html th,
            div.rendered_html td {
            text-align: right;
            vertical-align: middle;
            padding: 0.5em 0.5em;
            line-height: normal;
            white-space: normal;
            max-width: none;
            border: none;
            }
            div.rendered_html th {
            font-weight: bold;
            }
            div.rendered_html tbody tr:nth-child(odd) {
            background: #f5f5f5;
            }
            div.rendered_html tbody tr:hover {
            background: rgba(66, 165, 245, 0.2);
            }
        </style>
    </head>
    <body>
        <script src="https://unpkg.com/@jupyter-widgets/html-manager@^0.20.1/dist/embed.js" crossorigin="anonymous"></script>
    

    <script type="application/vnd.jupyter.widget-state+json">
    {"version_major": 2, "version_minor": 0, "state": {"77676d703b884d31814121dd16d816e0": {"model_name": "VBoxModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": ["rendered_html", "jp-RenderedHTMLCommon", "jp-RenderedHTML", "jp-OutputArea-output"], "children": ["IPY_MODEL_83a94bcdaa4d459da5e8d6c25685ba3b", "IPY_MODEL_0bd9c1c816044771909daa51b02ef012", "IPY_MODEL_d719bc7e8b0e492eab13be0ec2f1c352", "IPY_MODEL_be11a1817dcb44f8915c5bb64bcf2a30"], "layout": "IPY_MODEL_d0433e67fa0e40cc89fc242efb1a7f40"}}, "83a94bcdaa4d459da5e8d6c25685ba3b": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_4809bca17b324e4d9f305037bd748013", "style": "IPY_MODEL_c89402de1b5e48a0821cd90615e572ec", "value": "<h4><b>Data Duplicates</b></h4>"}}, "4809bca17b324e4d9f305037bd748013": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "c89402de1b5e48a0821cd90615e572ec": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "0bd9c1c816044771909daa51b02ef012": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_40a51ebaf76b4567ba1ffaa221108688", "style": "IPY_MODEL_c652c35aa3cc4eab9a0ad3ab98bfce3a", "value": "<p>Checks for duplicate samples in the dataset. <a href=\"https://docs.deepchecks.com/0.8/checks_gallery/tabular/data_integrity/plot_data_duplicates.html?utm_source=display_output&utm_medium=referral&utm_campaign=check_link\" target=\"_blank\">Read More...</a></p>"}}, "40a51ebaf76b4567ba1ffaa221108688": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "c652c35aa3cc4eab9a0ad3ab98bfce3a": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "d719bc7e8b0e492eab13be0ec2f1c352": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_e49320a2fbf24b13aa7de7b7dbb41bde", "style": "IPY_MODEL_c94089ad469f47d4b1bb2e7a1873af79"}}, "e49320a2fbf24b13aa7de7b7dbb41bde": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "c94089ad469f47d4b1bb2e7a1873af79": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "be11a1817dcb44f8915c5bb64bcf2a30": {"model_name": "VBoxModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "children": ["IPY_MODEL_7a41eb26832a4a62a09fdc93579fbe32", "IPY_MODEL_06be6a9cff8448a18dc52cc79ce425b9", "IPY_MODEL_d18023c4b3f64d48b513a73d9598a6ea", "IPY_MODEL_586f93f3d1234f67b2612f4c71494679"], "layout": "IPY_MODEL_d80c882e4ae94c0b9bd188f8e7c1ad5a"}}, "7a41eb26832a4a62a09fdc93579fbe32": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_92116c8144ab45f48ff66473aef32250", "style": "IPY_MODEL_170666be08394e80ad22779a10762fcf", "value": "<h5><b>Additional Outputs</b></h5>"}}, "92116c8144ab45f48ff66473aef32250": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "170666be08394e80ad22779a10762fcf": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "06be6a9cff8448a18dc52cc79ce425b9": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_bb43e302894e4faab50e0a996e026c11", "style": "IPY_MODEL_9b487b45aa0d401eb8749646fcb69b32", "value": "<div>0.22% of data samples are duplicates. </div>"}}, "bb43e302894e4faab50e0a996e026c11": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "9b487b45aa0d401eb8749646fcb69b32": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "d18023c4b3f64d48b513a73d9598a6ea": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_e040964309954f05adaab7f8d76a54eb", "style": "IPY_MODEL_38e627ea02d74073806f1c2e40fd9f21", "value": "<div>Each row in the table shows an example of duplicate data and the number of times it appears.</div>"}}, "e040964309954f05adaab7f8d76a54eb": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "38e627ea02d74073806f1c2e40fd9f21": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "586f93f3d1234f67b2612f4c71494679": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_d860bc0fc183451cafe6160991ea75cf", "style": "IPY_MODEL_4184983f661643148fb4cc35dec1ff0c", "value": "<style type=\"text/css\">\n#T_e589c_ table {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_e589c_ thead {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_e589c_ tbody {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_e589c_ th {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_e589c_ td {\n  text-align: left;\n  white-space: pre-wrap;\n}\n</style>\n<table id=\"T_e589c_\">\n  <thead>\n    <tr>\n      <th class=\"blank\" >&nbsp;</th>\n      <th class=\"blank level0\" >&nbsp;</th>\n      <th class=\"col_heading level0 col0\" >target</th>\n      <th class=\"col_heading level0 col1\" >month</th>\n      <th class=\"col_heading level0 col2\" >ext</th>\n      <th class=\"col_heading level0 col3\" >urlLength</th>\n      <th class=\"col_heading level0 col4\" >numDigits</th>\n      <th class=\"col_heading level0 col5\" >numParams</th>\n      <th class=\"col_heading level0 col6\" >num_%20</th>\n      <th class=\"col_heading level0 col7\" >num_@</th>\n      <th class=\"col_heading level0 col8\" >entropy</th>\n      <th class=\"col_heading level0 col9\" >has_ip</th>\n      <th class=\"col_heading level0 col10\" >hasHttp</th>\n      <th class=\"col_heading level0 col11\" >hasHttps</th>\n      <th class=\"col_heading level0 col12\" >urlIsLive</th>\n      <th class=\"col_heading level0 col13\" >dsr</th>\n      <th class=\"col_heading level0 col14\" >dse</th>\n      <th class=\"col_heading level0 col15\" >bodyLength</th>\n      <th class=\"col_heading level0 col16\" >numTitles</th>\n      <th class=\"col_heading level0 col17\" >numImages</th>\n      <th class=\"col_heading level0 col18\" >numLinks</th>\n      <th class=\"col_heading level0 col19\" >specialChars</th>\n      <th class=\"col_heading level0 col20\" >scriptLength</th>\n      <th class=\"col_heading level0 col21\" >sbr</th>\n      <th class=\"col_heading level0 col22\" >bscr</th>\n      <th class=\"col_heading level0 col23\" >sscr</th>\n    </tr>\n    <tr>\n      <th class=\"index_name level0\" >Instances</th>\n      <th class=\"index_name level1\" >Number of Duplicates</th>\n      <th class=\"blank col0\" >&nbsp;</th>\n      <th class=\"blank col1\" >&nbsp;</th>\n      <th class=\"blank col2\" >&nbsp;</th>\n      <th class=\"blank col3\" >&nbsp;</th>\n      <th class=\"blank col4\" >&nbsp;</th>\n      <th class=\"blank col5\" >&nbsp;</th>\n      <th class=\"blank col6\" >&nbsp;</th>\n      <th class=\"blank col7\" >&nbsp;</th>\n      <th class=\"blank col8\" >&nbsp;</th>\n      <th class=\"blank col9\" >&nbsp;</th>\n      <th class=\"blank col10\" >&nbsp;</th>\n      <th class=\"blank col11\" >&nbsp;</th>\n      <th class=\"blank col12\" >&nbsp;</th>\n      <th class=\"blank col13\" >&nbsp;</th>\n      <th class=\"blank col14\" >&nbsp;</th>\n      <th class=\"blank col15\" >&nbsp;</th>\n      <th class=\"blank col16\" >&nbsp;</th>\n      <th class=\"blank col17\" >&nbsp;</th>\n      <th class=\"blank col18\" >&nbsp;</th>\n      <th class=\"blank col19\" >&nbsp;</th>\n      <th class=\"blank col20\" >&nbsp;</th>\n      <th class=\"blank col21\" >&nbsp;</th>\n      <th class=\"blank col22\" >&nbsp;</th>\n      <th class=\"blank col23\" >&nbsp;</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th id=\"T_e589c_level0_row0\" class=\"row_heading level0 row0\" >4696, 4719, 5398</th>\n      <th id=\"T_e589c_level1_row0\" class=\"row_heading level1 row0\" >3</th>\n      <td id=\"T_e589c_row0_col0\" class=\"data row0 col0\" >0</td>\n      <td id=\"T_e589c_row0_col1\" class=\"data row0 col1\" >6</td>\n      <td id=\"T_e589c_row0_col2\" class=\"data row0 col2\" >other</td>\n      <td id=\"T_e589c_row0_col3\" class=\"data row0 col3\" >123</td>\n      <td id=\"T_e589c_row0_col4\" class=\"data row0 col4\" >28</td>\n      <td id=\"T_e589c_row0_col5\" class=\"data row0 col5\" >4</td>\n      <td id=\"T_e589c_row0_col6\" class=\"data row0 col6\" >0</td>\n      <td id=\"T_e589c_row0_col7\" class=\"data row0 col7\" >0</td>\n      <td id=\"T_e589c_row0_col8\" class=\"data row0 col8\" >-4.91</td>\n      <td id=\"T_e589c_row0_col9\" class=\"data row0 col9\" >0</td>\n      <td id=\"T_e589c_row0_col10\" class=\"data row0 col10\" >True</td>\n      <td id=\"T_e589c_row0_col11\" class=\"data row0 col11\" >False</td>\n      <td id=\"T_e589c_row0_col12\" class=\"data row0 col12\" >False</td>\n      <td id=\"T_e589c_row0_col13\" class=\"data row0 col13\" >0</td>\n      <td id=\"T_e589c_row0_col14\" class=\"data row0 col14\" >0</td>\n      <td id=\"T_e589c_row0_col15\" class=\"data row0 col15\" >0</td>\n      <td id=\"T_e589c_row0_col16\" class=\"data row0 col16\" >0</td>\n      <td id=\"T_e589c_row0_col17\" class=\"data row0 col17\" >0</td>\n      <td id=\"T_e589c_row0_col18\" class=\"data row0 col18\" >0</td>\n      <td id=\"T_e589c_row0_col19\" class=\"data row0 col19\" >0</td>\n      <td id=\"T_e589c_row0_col20\" class=\"data row0 col20\" >0</td>\n      <td id=\"T_e589c_row0_col21\" class=\"data row0 col21\" >0.00</td>\n      <td id=\"T_e589c_row0_col22\" class=\"data row0 col22\" >0.00</td>\n      <td id=\"T_e589c_row0_col23\" class=\"data row0 col23\" >0.00</td>\n    </tr>\n    <tr>\n      <th id=\"T_e589c_level0_row1\" class=\"row_heading level0 row1\" >82, 11342</th>\n      <th id=\"T_e589c_level1_row1\" class=\"row_heading level1 row1\" >2</th>\n      <td id=\"T_e589c_row1_col0\" class=\"data row1 col0\" >0</td>\n      <td id=\"T_e589c_row1_col1\" class=\"data row1 col1\" >1</td>\n      <td id=\"T_e589c_row1_col2\" class=\"data row1 col2\" >html</td>\n      <td id=\"T_e589c_row1_col3\" class=\"data row1 col3\" >92</td>\n      <td id=\"T_e589c_row1_col4\" class=\"data row1 col4\" >2</td>\n      <td id=\"T_e589c_row1_col5\" class=\"data row1 col5\" >0</td>\n      <td id=\"T_e589c_row1_col6\" class=\"data row1 col6\" >0</td>\n      <td id=\"T_e589c_row1_col7\" class=\"data row1 col7\" >0</td>\n      <td id=\"T_e589c_row1_col8\" class=\"data row1 col8\" >-4.31</td>\n      <td id=\"T_e589c_row1_col9\" class=\"data row1 col9\" >0</td>\n      <td id=\"T_e589c_row1_col10\" class=\"data row1 col10\" >True</td>\n      <td id=\"T_e589c_row1_col11\" class=\"data row1 col11\" >False</td>\n      <td id=\"T_e589c_row1_col12\" class=\"data row1 col12\" >False</td>\n      <td id=\"T_e589c_row1_col13\" class=\"data row1 col13\" >0</td>\n      <td id=\"T_e589c_row1_col14\" class=\"data row1 col14\" >0</td>\n      <td id=\"T_e589c_row1_col15\" class=\"data row1 col15\" >149</td>\n      <td id=\"T_e589c_row1_col16\" class=\"data row1 col16\" >1</td>\n      <td id=\"T_e589c_row1_col17\" class=\"data row1 col17\" >0</td>\n      <td id=\"T_e589c_row1_col18\" class=\"data row1 col18\" >0</td>\n      <td id=\"T_e589c_row1_col19\" class=\"data row1 col19\" >25</td>\n      <td id=\"T_e589c_row1_col20\" class=\"data row1 col20\" >0</td>\n      <td id=\"T_e589c_row1_col21\" class=\"data row1 col21\" >0.00</td>\n      <td id=\"T_e589c_row1_col22\" class=\"data row1 col22\" >0.17</td>\n      <td id=\"T_e589c_row1_col23\" class=\"data row1 col23\" >0.00</td>\n    </tr>\n    <tr>\n      <th id=\"T_e589c_level0_row2\" class=\"row_heading level0 row2\" >250, 790</th>\n      <th id=\"T_e589c_level1_row2\" class=\"row_heading level1 row2\" >2</th>\n      <td id=\"T_e589c_row2_col0\" class=\"data row2 col0\" >0</td>\n      <td id=\"T_e589c_row2_col1\" class=\"data row2 col1\" >1</td>\n      <td id=\"T_e589c_row2_col2\" class=\"data row2 col2\" >php</td>\n      <td id=\"T_e589c_row2_col3\" class=\"data row2 col3\" >107</td>\n      <td id=\"T_e589c_row2_col4\" class=\"data row2 col4\" >4</td>\n      <td id=\"T_e589c_row2_col5\" class=\"data row2 col5\" >8</td>\n      <td id=\"T_e589c_row2_col6\" class=\"data row2 col6\" >0</td>\n      <td id=\"T_e589c_row2_col7\" class=\"data row2 col7\" >0</td>\n      <td id=\"T_e589c_row2_col8\" class=\"data row2 col8\" >-4.53</td>\n      <td id=\"T_e589c_row2_col9\" class=\"data row2 col9\" >0</td>\n      <td id=\"T_e589c_row2_col10\" class=\"data row2 col10\" >True</td>\n      <td id=\"T_e589c_row2_col11\" class=\"data row2 col11\" >False</td>\n      <td id=\"T_e589c_row2_col12\" class=\"data row2 col12\" >False</td>\n      <td id=\"T_e589c_row2_col13\" class=\"data row2 col13\" >1381</td>\n      <td id=\"T_e589c_row2_col14\" class=\"data row2 col14\" >79</td>\n      <td id=\"T_e589c_row2_col15\" class=\"data row2 col15\" >0</td>\n      <td id=\"T_e589c_row2_col16\" class=\"data row2 col16\" >1</td>\n      <td id=\"T_e589c_row2_col17\" class=\"data row2 col17\" >0</td>\n      <td id=\"T_e589c_row2_col18\" class=\"data row2 col18\" >0</td>\n      <td id=\"T_e589c_row2_col19\" class=\"data row2 col19\" >0</td>\n      <td id=\"T_e589c_row2_col20\" class=\"data row2 col20\" >0</td>\n      <td id=\"T_e589c_row2_col21\" class=\"data row2 col21\" >0.00</td>\n      <td id=\"T_e589c_row2_col22\" class=\"data row2 col22\" >0.00</td>\n      <td id=\"T_e589c_row2_col23\" class=\"data row2 col23\" >0.00</td>\n    </tr>\n    <tr>\n      <th id=\"T_e589c_level0_row3\" class=\"row_heading level0 row3\" >6, 217</th>\n      <th id=\"T_e589c_level1_row3\" class=\"row_heading level1 row3\" >2</th>\n      <td id=\"T_e589c_row3_col0\" class=\"data row3 col0\" >0</td>\n      <td id=\"T_e589c_row3_col1\" class=\"data row3 col1\" >1</td>\n      <td id=\"T_e589c_row3_col2\" class=\"data row3 col2\" >php</td>\n      <td id=\"T_e589c_row3_col3\" class=\"data row3 col3\" >107</td>\n      <td id=\"T_e589c_row3_col4\" class=\"data row3 col4\" >5</td>\n      <td id=\"T_e589c_row3_col5\" class=\"data row3 col5\" >8</td>\n      <td id=\"T_e589c_row3_col6\" class=\"data row3 col6\" >0</td>\n      <td id=\"T_e589c_row3_col7\" class=\"data row3 col7\" >0</td>\n      <td id=\"T_e589c_row3_col8\" class=\"data row3 col8\" >-4.52</td>\n      <td id=\"T_e589c_row3_col9\" class=\"data row3 col9\" >0</td>\n      <td id=\"T_e589c_row3_col10\" class=\"data row3 col10\" >True</td>\n      <td id=\"T_e589c_row3_col11\" class=\"data row3 col11\" >False</td>\n      <td id=\"T_e589c_row3_col12\" class=\"data row3 col12\" >False</td>\n      <td id=\"T_e589c_row3_col13\" class=\"data row3 col13\" >1381</td>\n      <td id=\"T_e589c_row3_col14\" class=\"data row3 col14\" >79</td>\n      <td id=\"T_e589c_row3_col15\" class=\"data row3 col15\" >0</td>\n      <td id=\"T_e589c_row3_col16\" class=\"data row3 col16\" >1</td>\n      <td id=\"T_e589c_row3_col17\" class=\"data row3 col17\" >0</td>\n      <td id=\"T_e589c_row3_col18\" class=\"data row3 col18\" >0</td>\n      <td id=\"T_e589c_row3_col19\" class=\"data row3 col19\" >0</td>\n      <td id=\"T_e589c_row3_col20\" class=\"data row3 col20\" >0</td>\n      <td id=\"T_e589c_row3_col21\" class=\"data row3 col21\" >0.00</td>\n      <td id=\"T_e589c_row3_col22\" class=\"data row3 col22\" >0.00</td>\n      <td id=\"T_e589c_row3_col23\" class=\"data row3 col23\" >0.00</td>\n    </tr>\n    <tr>\n      <th id=\"T_e589c_level0_row4\" class=\"row_heading level0 row4\" >609, 763</th>\n      <th id=\"T_e589c_level1_row4\" class=\"row_heading level1 row4\" >2</th>\n      <td id=\"T_e589c_row4_col0\" class=\"data row4 col0\" >0</td>\n      <td id=\"T_e589c_row4_col1\" class=\"data row4 col1\" >1</td>\n      <td id=\"T_e589c_row4_col2\" class=\"data row4 col2\" >php</td>\n      <td id=\"T_e589c_row4_col3\" class=\"data row4 col3\" >113</td>\n      <td id=\"T_e589c_row4_col4\" class=\"data row4 col4\" >6</td>\n      <td id=\"T_e589c_row4_col5\" class=\"data row4 col5\" >8</td>\n      <td id=\"T_e589c_row4_col6\" class=\"data row4 col6\" >0</td>\n      <td id=\"T_e589c_row4_col7\" class=\"data row4 col7\" >0</td>\n      <td id=\"T_e589c_row4_col8\" class=\"data row4 col8\" >-4.63</td>\n      <td id=\"T_e589c_row4_col9\" class=\"data row4 col9\" >0</td>\n      <td id=\"T_e589c_row4_col10\" class=\"data row4 col10\" >True</td>\n      <td id=\"T_e589c_row4_col11\" class=\"data row4 col11\" >False</td>\n      <td id=\"T_e589c_row4_col12\" class=\"data row4 col12\" >False</td>\n      <td id=\"T_e589c_row4_col13\" class=\"data row4 col13\" >1381</td>\n      <td id=\"T_e589c_row4_col14\" class=\"data row4 col14\" >79</td>\n      <td id=\"T_e589c_row4_col15\" class=\"data row4 col15\" >0</td>\n      <td id=\"T_e589c_row4_col16\" class=\"data row4 col16\" >1</td>\n      <td id=\"T_e589c_row4_col17\" class=\"data row4 col17\" >0</td>\n      <td id=\"T_e589c_row4_col18\" class=\"data row4 col18\" >0</td>\n      <td id=\"T_e589c_row4_col19\" class=\"data row4 col19\" >0</td>\n      <td id=\"T_e589c_row4_col20\" class=\"data row4 col20\" >0</td>\n      <td id=\"T_e589c_row4_col21\" class=\"data row4 col21\" >0.00</td>\n      <td id=\"T_e589c_row4_col22\" class=\"data row4 col22\" >0.00</td>\n      <td id=\"T_e589c_row4_col23\" class=\"data row4 col23\" >0.00</td>\n    </tr>\n    <tr>\n      <th id=\"T_e589c_level0_row5\" class=\"row_heading level0 row5\" >974, 1557</th>\n      <th id=\"T_e589c_level1_row5\" class=\"row_heading level1 row5\" >2</th>\n      <td id=\"T_e589c_row5_col0\" class=\"data row5 col0\" >0</td>\n      <td id=\"T_e589c_row5_col1\" class=\"data row5 col1\" >2</td>\n      <td id=\"T_e589c_row5_col2\" class=\"data row5 col2\" >html</td>\n      <td id=\"T_e589c_row5_col3\" class=\"data row5 col3\" >92</td>\n      <td id=\"T_e589c_row5_col4\" class=\"data row5 col4\" >2</td>\n      <td id=\"T_e589c_row5_col5\" class=\"data row5 col5\" >0</td>\n      <td id=\"T_e589c_row5_col6\" class=\"data row5 col6\" >0</td>\n      <td id=\"T_e589c_row5_col7\" class=\"data row5 col7\" >0</td>\n      <td id=\"T_e589c_row5_col8\" class=\"data row5 col8\" >-4.31</td>\n      <td id=\"T_e589c_row5_col9\" class=\"data row5 col9\" >0</td>\n      <td id=\"T_e589c_row5_col10\" class=\"data row5 col10\" >True</td>\n      <td id=\"T_e589c_row5_col11\" class=\"data row5 col11\" >False</td>\n      <td id=\"T_e589c_row5_col12\" class=\"data row5 col12\" >False</td>\n      <td id=\"T_e589c_row5_col13\" class=\"data row5 col13\" >0</td>\n      <td id=\"T_e589c_row5_col14\" class=\"data row5 col14\" >0</td>\n      <td id=\"T_e589c_row5_col15\" class=\"data row5 col15\" >149</td>\n      <td id=\"T_e589c_row5_col16\" class=\"data row5 col16\" >1</td>\n      <td id=\"T_e589c_row5_col17\" class=\"data row5 col17\" >0</td>\n      <td id=\"T_e589c_row5_col18\" class=\"data row5 col18\" >0</td>\n      <td id=\"T_e589c_row5_col19\" class=\"data row5 col19\" >25</td>\n      <td id=\"T_e589c_row5_col20\" class=\"data row5 col20\" >0</td>\n      <td id=\"T_e589c_row5_col21\" class=\"data row5 col21\" >0.00</td>\n      <td id=\"T_e589c_row5_col22\" class=\"data row5 col22\" >0.17</td>\n      <td id=\"T_e589c_row5_col23\" class=\"data row5 col23\" >0.00</td>\n    </tr>\n    <tr>\n      <th id=\"T_e589c_level0_row6\" class=\"row_heading level0 row6\" >2150, 2360</th>\n      <th id=\"T_e589c_level1_row6\" class=\"row_heading level1 row6\" >2</th>\n      <td id=\"T_e589c_row6_col0\" class=\"data row6 col0\" >0</td>\n      <td id=\"T_e589c_row6_col1\" class=\"data row6 col1\" >3</td>\n      <td id=\"T_e589c_row6_col2\" class=\"data row6 col2\" >html</td>\n      <td id=\"T_e589c_row6_col3\" class=\"data row6 col3\" >92</td>\n      <td id=\"T_e589c_row6_col4\" class=\"data row6 col4\" >2</td>\n      <td id=\"T_e589c_row6_col5\" class=\"data row6 col5\" >0</td>\n      <td id=\"T_e589c_row6_col6\" class=\"data row6 col6\" >0</td>\n      <td id=\"T_e589c_row6_col7\" class=\"data row6 col7\" >0</td>\n      <td id=\"T_e589c_row6_col8\" class=\"data row6 col8\" >-4.31</td>\n      <td id=\"T_e589c_row6_col9\" class=\"data row6 col9\" >0</td>\n      <td id=\"T_e589c_row6_col10\" class=\"data row6 col10\" >True</td>\n      <td id=\"T_e589c_row6_col11\" class=\"data row6 col11\" >False</td>\n      <td id=\"T_e589c_row6_col12\" class=\"data row6 col12\" >False</td>\n      <td id=\"T_e589c_row6_col13\" class=\"data row6 col13\" >0</td>\n      <td id=\"T_e589c_row6_col14\" class=\"data row6 col14\" >0</td>\n      <td id=\"T_e589c_row6_col15\" class=\"data row6 col15\" >149</td>\n      <td id=\"T_e589c_row6_col16\" class=\"data row6 col16\" >1</td>\n      <td id=\"T_e589c_row6_col17\" class=\"data row6 col17\" >0</td>\n      <td id=\"T_e589c_row6_col18\" class=\"data row6 col18\" >0</td>\n      <td id=\"T_e589c_row6_col19\" class=\"data row6 col19\" >25</td>\n      <td id=\"T_e589c_row6_col20\" class=\"data row6 col20\" >0</td>\n      <td id=\"T_e589c_row6_col21\" class=\"data row6 col21\" >0.00</td>\n      <td id=\"T_e589c_row6_col22\" class=\"data row6 col22\" >0.17</td>\n      <td id=\"T_e589c_row6_col23\" class=\"data row6 col23\" >0.00</td>\n    </tr>\n    <tr>\n      <th id=\"T_e589c_level0_row7\" class=\"row_heading level0 row7\" >2238, 2489</th>\n      <th id=\"T_e589c_level1_row7\" class=\"row_heading level1 row7\" >2</th>\n      <td id=\"T_e589c_row7_col0\" class=\"data row7 col0\" >0</td>\n      <td id=\"T_e589c_row7_col1\" class=\"data row7 col1\" >3</td>\n      <td id=\"T_e589c_row7_col2\" class=\"data row7 col2\" >php</td>\n      <td id=\"T_e589c_row7_col3\" class=\"data row7 col3\" >108</td>\n      <td id=\"T_e589c_row7_col4\" class=\"data row7 col4\" >3</td>\n      <td id=\"T_e589c_row7_col5\" class=\"data row7 col5\" >8</td>\n      <td id=\"T_e589c_row7_col6\" class=\"data row7 col6\" >0</td>\n      <td id=\"T_e589c_row7_col7\" class=\"data row7 col7\" >0</td>\n      <td id=\"T_e589c_row7_col8\" class=\"data row7 col8\" >-4.51</td>\n      <td id=\"T_e589c_row7_col9\" class=\"data row7 col9\" >0</td>\n      <td id=\"T_e589c_row7_col10\" class=\"data row7 col10\" >True</td>\n      <td id=\"T_e589c_row7_col11\" class=\"data row7 col11\" >False</td>\n      <td id=\"T_e589c_row7_col12\" class=\"data row7 col12\" >False</td>\n      <td id=\"T_e589c_row7_col13\" class=\"data row7 col13\" >1381</td>\n      <td id=\"T_e589c_row7_col14\" class=\"data row7 col14\" >79</td>\n      <td id=\"T_e589c_row7_col15\" class=\"data row7 col15\" >0</td>\n      <td id=\"T_e589c_row7_col16\" class=\"data row7 col16\" >1</td>\n      <td id=\"T_e589c_row7_col17\" class=\"data row7 col17\" >0</td>\n      <td id=\"T_e589c_row7_col18\" class=\"data row7 col18\" >0</td>\n      <td id=\"T_e589c_row7_col19\" class=\"data row7 col19\" >0</td>\n      <td id=\"T_e589c_row7_col20\" class=\"data row7 col20\" >0</td>\n      <td id=\"T_e589c_row7_col21\" class=\"data row7 col21\" >0.00</td>\n      <td id=\"T_e589c_row7_col22\" class=\"data row7 col22\" >0.00</td>\n      <td id=\"T_e589c_row7_col23\" class=\"data row7 col23\" >0.00</td>\n    </tr>\n    <tr>\n      <th id=\"T_e589c_level0_row8\" class=\"row_heading level0 row8\" >3192, 3444</th>\n      <th id=\"T_e589c_level1_row8\" class=\"row_heading level1 row8\" >2</th>\n      <td id=\"T_e589c_row8_col0\" class=\"data row8 col0\" >0</td>\n      <td id=\"T_e589c_row8_col1\" class=\"data row8 col1\" >4</td>\n      <td id=\"T_e589c_row8_col2\" class=\"data row8 col2\" >other</td>\n      <td id=\"T_e589c_row8_col3\" class=\"data row8 col3\" >123</td>\n      <td id=\"T_e589c_row8_col4\" class=\"data row8 col4\" >28</td>\n      <td id=\"T_e589c_row8_col5\" class=\"data row8 col5\" >4</td>\n      <td id=\"T_e589c_row8_col6\" class=\"data row8 col6\" >0</td>\n      <td id=\"T_e589c_row8_col7\" class=\"data row8 col7\" >0</td>\n      <td id=\"T_e589c_row8_col8\" class=\"data row8 col8\" >-4.92</td>\n      <td id=\"T_e589c_row8_col9\" class=\"data row8 col9\" >0</td>\n      <td id=\"T_e589c_row8_col10\" class=\"data row8 col10\" >True</td>\n      <td id=\"T_e589c_row8_col11\" class=\"data row8 col11\" >False</td>\n      <td id=\"T_e589c_row8_col12\" class=\"data row8 col12\" >False</td>\n      <td id=\"T_e589c_row8_col13\" class=\"data row8 col13\" >0</td>\n      <td id=\"T_e589c_row8_col14\" class=\"data row8 col14\" >0</td>\n      <td id=\"T_e589c_row8_col15\" class=\"data row8 col15\" >0</td>\n      <td id=\"T_e589c_row8_col16\" class=\"data row8 col16\" >0</td>\n      <td id=\"T_e589c_row8_col17\" class=\"data row8 col17\" >0</td>\n      <td id=\"T_e589c_row8_col18\" class=\"data row8 col18\" >0</td>\n      <td id=\"T_e589c_row8_col19\" class=\"data row8 col19\" >0</td>\n      <td id=\"T_e589c_row8_col20\" class=\"data row8 col20\" >0</td>\n      <td id=\"T_e589c_row8_col21\" class=\"data row8 col21\" >0.00</td>\n      <td id=\"T_e589c_row8_col22\" class=\"data row8 col22\" >0.00</td>\n      <td id=\"T_e589c_row8_col23\" class=\"data row8 col23\" >0.00</td>\n    </tr>\n    <tr>\n      <th id=\"T_e589c_level0_row9\" class=\"row_heading level0 row9\" >3277, 3498</th>\n      <th id=\"T_e589c_level1_row9\" class=\"row_heading level1 row9\" >2</th>\n      <td id=\"T_e589c_row9_col0\" class=\"data row9 col0\" >0</td>\n      <td id=\"T_e589c_row9_col1\" class=\"data row9 col1\" >4</td>\n      <td id=\"T_e589c_row9_col2\" class=\"data row9 col2\" >php</td>\n      <td id=\"T_e589c_row9_col3\" class=\"data row9 col3\" >93</td>\n      <td id=\"T_e589c_row9_col4\" class=\"data row9 col4\" >31</td>\n      <td id=\"T_e589c_row9_col5\" class=\"data row9 col5\" >1</td>\n      <td id=\"T_e589c_row9_col6\" class=\"data row9 col6\" >0</td>\n      <td id=\"T_e589c_row9_col7\" class=\"data row9 col7\" >0</td>\n      <td id=\"T_e589c_row9_col8\" class=\"data row9 col8\" >-4.93</td>\n      <td id=\"T_e589c_row9_col9\" class=\"data row9 col9\" >0</td>\n      <td id=\"T_e589c_row9_col10\" class=\"data row9 col10\" >True</td>\n      <td id=\"T_e589c_row9_col11\" class=\"data row9 col11\" >False</td>\n      <td id=\"T_e589c_row9_col12\" class=\"data row9 col12\" >False</td>\n      <td id=\"T_e589c_row9_col13\" class=\"data row9 col13\" >0</td>\n      <td id=\"T_e589c_row9_col14\" class=\"data row9 col14\" >0</td>\n      <td id=\"T_e589c_row9_col15\" class=\"data row9 col15\" >281</td>\n      <td id=\"T_e589c_row9_col16\" class=\"data row9 col16\" >0</td>\n      <td id=\"T_e589c_row9_col17\" class=\"data row9 col17\" >0</td>\n      <td id=\"T_e589c_row9_col18\" class=\"data row9 col18\" >0</td>\n      <td id=\"T_e589c_row9_col19\" class=\"data row9 col19\" >74</td>\n      <td id=\"T_e589c_row9_col20\" class=\"data row9 col20\" >142</td>\n      <td id=\"T_e589c_row9_col21\" class=\"data row9 col21\" >0.51</td>\n      <td id=\"T_e589c_row9_col22\" class=\"data row9 col22\" >0.26</td>\n      <td id=\"T_e589c_row9_col23\" class=\"data row9 col23\" >1.92</td>\n    </tr>\n  </tbody>\n</table>\n"}}, "d860bc0fc183451cafe6160991ea75cf": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "4184983f661643148fb4cc35dec1ff0c": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "d80c882e4ae94c0b9bd188f8e7c1ad5a": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "d0433e67fa0e40cc89fc242efb1a7f40": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}}}
    </script>
    <script type="application/vnd.jupyter.widget-view+json">
    {"version_major": 2, "version_minor": 0, "model_id": "77676d703b884d31814121dd16d816e0"}
    </script>

    </body>
    </html>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 63-68

Define a Condition
==================
Now, we define a condition that enforce the ratio of duplicates to be 0. A condition
is deepchecks' way to validate model and data quality, and let you know if anything
goes wrong.

.. GENERATED FROM PYTHON SOURCE LINES 68-73

.. code-block:: default


    check = DataDuplicates()
    check.add_condition_ratio_less_or_equal(0)
    result = check.run(phishing_dataset)
    result.show(show_additional_outputs=False)





.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <!--
        ~ ----------------------------------------------------------------------------
        ~ Copyright (C) 2021-2022 Deepchecks (https://www.deepchecks.com)
        ~
        ~ This file is part of Deepchecks.
        ~ Deepchecks is distributed under the terms of the GNU Affero General
        ~ Public License (version 3 or later).
        ~ You should have received a copy of the GNU Affero General Public License
        ~ along with Deepchecks.  If not, see <http://www.gnu.org/licenses/>.
        ~ ----------------------------------------------------------------------------
        ~
    -->
    <!DOCTYPE html>
    <html lang="en">
    <head>
        <meta charset="UTF-8">
        <title>Data Duplicates</title>
        <style>
            body {
                font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Helvetica, Arial, sans-serif, 'Apple Color Emoji', 'Segoe UI Emoji', 'Segoe UI Symbol';
                font-size: 1rem;
                line-height: 1.5;
                color: #212529;
                text-align: left;
                max-width: 1200px;
                margin: auto;
                background-color: white;
            }

            div.nbinput.container div.prompt *,
            div.nboutput.container div.prompt *,
            div.nbinput.container div.input_area pre,
            div.nboutput.container div.output_area pre,
            div.nbinput.container div.input_area .highlight,
            div.nboutput.container div.output_area .highlight {
                border: none;
                padding: 0;
                margin: 0;
                box-shadow: none;
            }

            div.nbinput.container > div[class*=highlight],
            div.nboutput.container > div[class*=highlight] {
                margin: 0;
            }

            div.nbinput.container div.prompt *,
            div.nboutput.container div.prompt * {
                background: none;
            }

            div.nboutput.container div.output_area .highlight,
            div.nboutput.container div.output_area pre {
                background: unset;
            }

            div.nboutput.container div.output_area div.highlight {
                color: unset;  /* override Pygments text color */
            }

            /* avoid gaps between output lines */
            div.nboutput.container div[class*=highlight] pre {
                line-height: normal;
            }

            /* input/output containers */
            div.nbinput.container,
            div.nboutput.container {
                display: -webkit-flex;
                display: flex;
                align-items: flex-start;
                margin: 0;
                width: 100%;
            }
            @media (max-width: 540px) {
                div.nbinput.container,
                div.nboutput.container {
                    flex-direction: column;
                }
            }

            /* input container */
            div.nbinput.container {
                padding-top: 5px;
            }

            /* last container */
            div.nblast.container {
                padding-bottom: 5px;
            }

            /* input prompt */
            div.nbinput.container div.prompt pre {
                color: #307FC1;
            }

            /* output prompt */
            div.nboutput.container div.prompt pre {
                color: #BF5B3D;
            }

            /* all prompts */
            div.nbinput.container div.prompt,
            div.nboutput.container div.prompt {
                width: 4.5ex;
                padding-top: 5px;
                position: relative;
                user-select: none;
            }

            div.nbinput.container div.prompt > div,
            div.nboutput.container div.prompt > div {
                position: absolute;
                right: 0;
                margin-right: 0.3ex;
            }

            @media (max-width: 540px) {
                div.nbinput.container div.prompt,
                div.nboutput.container div.prompt {
                    width: unset;
                    text-align: left;
                    padding: 0.4em;
                }
                div.nboutput.container div.prompt.empty {
                    padding: 0;
                }

                div.nbinput.container div.prompt > div,
                div.nboutput.container div.prompt > div {
                    position: unset;
                }
            }

            /* disable scrollbars on prompts */
            div.nbinput.container div.prompt pre,
            div.nboutput.container div.prompt pre {
                overflow: hidden;
            }

            /* input/output area */
            div.nbinput.container div.input_area,
            div.nboutput.container div.output_area {
                -webkit-flex: 1;
                flex: 1;
                overflow: auto;
            }
            @media (max-width: 540px) {
                div.nbinput.container div.input_area,
                div.nboutput.container div.output_area {
                    width: 100%;
                }
            }

            /* input area */
            div.nbinput.container div.input_area {
                border: 1px solid #e0e0e0;
                border-radius: 2px;
                /*background: #f5f5f5;*/
            }

            /* override MathJax center alignment in output cells */
            div.nboutput.container div[class*=MathJax] {
                text-align: left !important;
            }

            /* override sphinx.ext.imgmath center alignment in output cells */
            div.nboutput.container div.math p {
                text-align: left;
            }

            /* standard error */
            div.nboutput.container div.output_area.stderr {
                background: #fdd;
            }

            /* ANSI colors */
            .ansi-black-fg { color: #3E424D; }
            .ansi-black-bg { background-color: #3E424D; }
            .ansi-black-intense-fg { color: #282C36; }
            .ansi-black-intense-bg { background-color: #282C36; }
            .ansi-red-fg { color: #E75C58; }
            .ansi-red-bg { background-color: #E75C58; }
            .ansi-red-intense-fg { color: #B22B31; }
            .ansi-red-intense-bg { background-color: #B22B31; }
            .ansi-green-fg { color: #00A250; }
            .ansi-green-bg { background-color: #00A250; }
            .ansi-green-intense-fg { color: #007427; }
            .ansi-green-intense-bg { background-color: #007427; }
            .ansi-yellow-fg { color: #DDB62B; }
            .ansi-yellow-bg { background-color: #DDB62B; }
            .ansi-yellow-intense-fg { color: #B27D12; }
            .ansi-yellow-intense-bg { background-color: #B27D12; }
            .ansi-blue-fg { color: #208FFB; }
            .ansi-blue-bg { background-color: #208FFB; }
            .ansi-blue-intense-fg { color: #0065CA; }
            .ansi-blue-intense-bg { background-color: #0065CA; }
            .ansi-magenta-fg { color: #D160C4; }
            .ansi-magenta-bg { background-color: #D160C4; }
            .ansi-magenta-intense-fg { color: #A03196; }
            .ansi-magenta-intense-bg { background-color: #A03196; }
            .ansi-cyan-fg { color: #60C6C8; }
            .ansi-cyan-bg { background-color: #60C6C8; }
            .ansi-cyan-intense-fg { color: #258F8F; }
            .ansi-cyan-intense-bg { background-color: #258F8F; }
            .ansi-white-fg { color: #C5C1B4; }
            .ansi-white-bg { background-color: #C5C1B4; }
            .ansi-white-intense-fg { color: #A1A6B2; }
            .ansi-white-intense-bg { background-color: #A1A6B2; }

            .ansi-default-inverse-fg { color: #FFFFFF; }
            .ansi-default-inverse-bg { background-color: #000000; }

            .ansi-bold { font-weight: bold; }
            .ansi-underline { text-decoration: underline; }


            div.nbinput.container div.input_area div[class*=highlight] > pre,
            div.nboutput.container div.output_area div[class*=highlight] > pre,
            div.nboutput.container div.output_area div[class*=highlight].math,
            div.nboutput.container div.output_area.rendered_html,
            div.nboutput.container div.output_area > div.output_javascript,
            div.nboutput.container div.output_area:not(.rendered_html) > img{
                padding: 5px;
                margin: 0;
            }

            /* fix copybtn overflow problem in chromium (needed for 'sphinx_copybutton') */
            div.nbinput.container div.input_area > div[class^='highlight'],
            div.nboutput.container div.output_area > div[class^='highlight']{
                overflow-y: hidden;
            }

            /* hide copybtn icon on prompts (needed for 'sphinx_copybutton') */
            .prompt .copybtn {
                display: none;
            }

            /* Some additional styling taken form the Jupyter notebook CSS */
            div.rendered_html table {
            border: none;
            border-collapse: collapse;
            border-spacing: 0;
            color: black;
            font-size: 12px;
            table-layout: fixed;
            }
            div.rendered_html thead {
            border-bottom: 1px solid black;
            vertical-align: bottom;
            }
            div.rendered_html tr,
            div.rendered_html th,
            div.rendered_html td {
            text-align: right;
            vertical-align: middle;
            padding: 0.5em 0.5em;
            line-height: normal;
            white-space: normal;
            max-width: none;
            border: none;
            }
            div.rendered_html th {
            font-weight: bold;
            }
            div.rendered_html tbody tr:nth-child(odd) {
            background: #f5f5f5;
            }
            div.rendered_html tbody tr:hover {
            background: rgba(66, 165, 245, 0.2);
            }
        </style>
    </head>
    <body>
        <script src="https://unpkg.com/@jupyter-widgets/html-manager@^0.20.1/dist/embed.js" crossorigin="anonymous"></script>
    

    <script type="application/vnd.jupyter.widget-state+json">
    {"version_major": 2, "version_minor": 0, "state": {"f061475b3e8349e78c66aa7a0a109873": {"model_name": "VBoxModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": ["rendered_html", "jp-RenderedHTMLCommon", "jp-RenderedHTML", "jp-OutputArea-output"], "children": ["IPY_MODEL_9308809584ff4ec98c77bd39375d053c", "IPY_MODEL_315076d394494443afa9d19ec445751b", "IPY_MODEL_6024b73a98364c7a8dd697754b6b3c04"], "layout": "IPY_MODEL_c15e2707b7ad4864ab4983c374400fbb"}}, "9308809584ff4ec98c77bd39375d053c": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_bda1a6a808d647b8829199a55f62f883", "style": "IPY_MODEL_7e5c5d40d08047e58956cf4fcf1b2951", "value": "<h4><b>Data Duplicates</b></h4>"}}, "bda1a6a808d647b8829199a55f62f883": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "7e5c5d40d08047e58956cf4fcf1b2951": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "315076d394494443afa9d19ec445751b": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_53ad7dbef7a04df58541a8e09af75293", "style": "IPY_MODEL_9941d8c342ed4e5dabb79d3189066a3c", "value": "<p>Checks for duplicate samples in the dataset. <a href=\"https://docs.deepchecks.com/0.8/checks_gallery/tabular/data_integrity/plot_data_duplicates.html?utm_source=display_output&utm_medium=referral&utm_campaign=check_link\" target=\"_blank\">Read More...</a></p>"}}, "53ad7dbef7a04df58541a8e09af75293": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "9941d8c342ed4e5dabb79d3189066a3c": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "6024b73a98364c7a8dd697754b6b3c04": {"model_name": "HTMLModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_classes": [], "layout": "IPY_MODEL_0281100233f449ceace994e0e23081f5", "style": "IPY_MODEL_52aa886142cd4bf8b7420e4304ce434b", "value": "<h5><b>Conditions Summary</b></h5><style type=\"text/css\">\n#T_1e069_ table {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_1e069_ thead {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_1e069_ tbody {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_1e069_ th {\n  text-align: left;\n  white-space: pre-wrap;\n}\n#T_1e069_ td {\n  text-align: left;\n  white-space: pre-wrap;\n}\n</style>\n<table id=\"T_1e069_\">\n  <thead>\n    <tr>\n      <th class=\"col_heading level0 col0\" >Status</th>\n      <th class=\"col_heading level0 col1\" >Condition</th>\n      <th class=\"col_heading level0 col2\" >More Info</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <td id=\"T_1e069_row0_col0\" class=\"data row0 col0\" ><div style=\"color: orange;text-align: center;font-weight:bold\">!</div></td>\n      <td id=\"T_1e069_row0_col1\" class=\"data row0 col1\" >Duplicate data ratio is less or equal to 0%</td>\n      <td id=\"T_1e069_row0_col2\" class=\"data row0 col2\" >Found 0.0088% duplicate data</td>\n    </tr>\n  </tbody>\n</table>\n"}}, "0281100233f449ceace994e0e23081f5": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}, "52aa886142cd4bf8b7420e4304ce434b": {"model_name": "DescriptionStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {}}, "c15e2707b7ad4864ab4983c374400fbb": {"model_name": "LayoutModel", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {}}}}
    </script>
    <script type="application/vnd.jupyter.widget-view+json">
    {"version_major": 2, "version_minor": 0, "model_id": "f061475b3e8349e78c66aa7a0a109873"}
    </script>

    </body>
    </html>
    </div>
    <br />
    <br />


.. rst-class:: sphx-glr-timing

   **Total running time of the script:** ( 0 minutes  2.610 seconds)


.. _sphx_glr_download_checks_gallery_tabular_data_integrity_plot_data_duplicates.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example


    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: plot_data_duplicates.py <plot_data_duplicates.py>`

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: plot_data_duplicates.ipynb <plot_data_duplicates.ipynb>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_