.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "checks_gallery/vision/model_evaluation/plot_model_error_analysis.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note Click :ref:`here ` to download the full example code .. rst-class:: sphx-glr-example-title .. _sphx_glr_checks_gallery_vision_model_evaluation_plot_model_error_analysis.py: Model Error Analysis check ========================== This notebooks provides an overview for using and understanding the model error analysis check. **Structure:** - `What is the purpose of the check? <#what-is-the-purpose-of-the-check>`__ - `Classification <#classification-performance-report>`__ - `Generate data & model <#generate_c>`__ - `Run the check <#run_check_c>`__ - `Object Detection <#object-detection-class-performance>`__ - `Generate data & model <#generate_o>`__ - `Run the check <#run_check_o>`__ What is the purpose of the check? --------------------------------- .. GENERATED FROM PYTHON SOURCE LINES 28-30 Imports ------- .. GENERATED FROM PYTHON SOURCE LINES 30-32 .. code-block:: default from deepchecks.vision.checks import ModelErrorAnalysis .. GENERATED FROM PYTHON SOURCE LINES 33-40 Classification Performance Report --------------------------------- .. _generate_c: Generate data and model: ~~~~~~~~~~~~~~~~~~~~~~~~ .. GENERATED FROM PYTHON SOURCE LINES 40-46 .. code-block:: default from deepchecks.vision.datasets.classification import mnist mnist_model = mnist.load_model() train_ds = mnist.load_dataset(train=True, object_type='VisionData') test_ds = mnist.load_dataset(train=False, object_type='VisionData') .. GENERATED FROM PYTHON SOURCE LINES 47-51 .. _run_check_c: Run the check: ~~~~~~~~~~~~~~~~ .. GENERATED FROM PYTHON SOURCE LINES 51-54 .. code-block:: default check = ModelErrorAnalysis(min_error_model_score=-0.1) check.run(train_ds, test_ds, mnist_model) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none Validating Input: | | 0/1 [00:00 Model Error Analysis

.. GENERATED FROM PYTHON SOURCE LINES 55-62 Object Detection Class Performance ---------------------------------- For object detection tasks - the default metric that is being calculated it the Average Precision. The definition of the Average Precision is identical to how the COCO dataset defined it - mean of the average precision per class, over the range [0.5, 0.95, 0.05] of IoU thresholds. .. GENERATED FROM PYTHON SOURCE LINES 62-67 .. code-block:: default import numpy as np from deepchecks.vision.datasets.detection import coco .. GENERATED FROM PYTHON SOURCE LINES 68-76 .. _generate_o: Generate Data and Model ~~~~~~~~~~~~~~~~~~~~~~~ We generate a sample dataset of 128 images from the `COCO dataset `__, and using the `YOLOv5 model `__ .. GENERATED FROM PYTHON SOURCE LINES 76-82 .. code-block:: default yolo = coco.load_model(pretrained=True) train_ds = coco.load_dataset(train=True, object_type='VisionData') test_ds = coco.load_dataset(train=False, object_type='VisionData') .. GENERATED FROM PYTHON SOURCE LINES 83-87 .. _run_check_o: Run the check: ~~~~~~~~~~~~~~~~ .. GENERATED FROM PYTHON SOURCE LINES 87-90 .. code-block:: default check = ModelErrorAnalysis(min_error_model_score=-1) check.run(train_ds, test_ds, yolo) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none Validating Input: | | 0/1 [00:00 Model Error Analysis

.. rst-class:: sphx-glr-timing **Total running time of the script:** ( 1 minutes 14.945 seconds) .. _sphx_glr_download_checks_gallery_vision_model_evaluation_plot_model_error_analysis.py: .. only :: html .. container:: sphx-glr-footer :class: sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: plot_model_error_analysis.py ` .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: plot_model_error_analysis.ipynb ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_