.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "checks_gallery/vision/model_evaluation/plot_mean_average_recall_report.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note Click :ref:`here ` to download the full example code .. rst-class:: sphx-glr-example-title .. _sphx_glr_checks_gallery_vision_model_evaluation_plot_mean_average_recall_report.py: .. _plot_vision_mean_average_recall_report: Mean Average Recall Report ************************** This notebooks provides an overview for using and understanding the mean average recall report check. **Structure:** * `What is the purpose of the check? <#what-is-the-purpose-of-the-check>`__ * `Generate data & model <#generate-data-and-model>`__ * `Run the check <#run-the-check>`__ * `Define a condition <#define-a-condition>`__ What is the purpose of the check? ================================= The Mean Average Recall Report evaluates the `mAR metric `__ on the given model and data, and returns the mAR values per bounding box size category (small, medium, large). This check only works on the Object Detection task. .. GENERATED FROM PYTHON SOURCE LINES 26-28 Imports ------- .. GENERATED FROM PYTHON SOURCE LINES 28-34 .. code-block:: default import numpy as np from deepchecks.vision.checks import MeanAverageRecallReport from deepchecks.vision.datasets.detection import coco .. GENERATED FROM PYTHON SOURCE LINES 35-42 Generate Data and Model ----------------------- We generate a sample dataset of 128 images from the `COCO dataset `__, and using the `YOLOv5 model `__. For the label formatter - our dataset returns exactly the accepted format, so our formatting function is the simple `lambda x: x` function. .. GENERATED FROM PYTHON SOURCE LINES 42-47 .. code-block:: default yolo = coco.load_model(pretrained=True) test_ds = coco.load_dataset(train=False, object_type='VisionData') .. GENERATED FROM PYTHON SOURCE LINES 48-50 Run the check ------------- .. GENERATED FROM PYTHON SOURCE LINES 50-55 .. code-block:: default check = MeanAverageRecallReport() result = check.run(test_ds, yolo) result .. rst-class:: sphx-glr-script-out .. code-block:: none Validating Input: | | 0/1 [Time: 00:00] Validating Input: |#####| 1/1 [Time: 00:04] Validating Input: |#####| 1/1 [Time: 00:04] Ingesting Batches: | | 0/2 [Time: 00:00] Ingesting Batches: |##5 | 1/2 [Time: 00:04] Ingesting Batches: |#####| 2/2 [Time: 00:09] Ingesting Batches: |#####| 2/2 [Time: 00:09] Computing Check: | | 0/1 [Time: 00:00] Computing Check: |#####| 1/1 [Time: 00:00] Computing Check: |#####| 1/1 [Time: 00:00] .. raw:: html
Mean Average Recall Report


.. GENERATED FROM PYTHON SOURCE LINES 56-59 If you have a GPU, you can speed up this check by passing it as an argument to .run() as device= To display the results in an IDE like PyCharm, you can use the following code: .. GENERATED FROM PYTHON SOURCE LINES 59-61 .. code-block:: default # result.show_in_window() .. GENERATED FROM PYTHON SOURCE LINES 62-63 The result will be displayed in a new window. .. GENERATED FROM PYTHON SOURCE LINES 65-68 Observe the check’s output -------------------------- The result value is a dataframe that has the average recall score per each area range and IoU. .. GENERATED FROM PYTHON SOURCE LINES 68-71 .. code-block:: default result.value .. raw:: html
AR@1 (%) AR@10 (%) AR@100 (%)
Area size
All 0.330552 0.423444 0.429179
Small (area < 32^2) 0.104955 0.220594 0.220594
Medium (32^2 < area < 96^2) 0.325099 0.417392 0.423844
Large (area < 96^2) 0.481611 0.544408 0.549963


.. GENERATED FROM PYTHON SOURCE LINES 72-76 Define a condition ================== We can define a condition that checks whether our model's average recall score is not less than a given threshold .. GENERATED FROM PYTHON SOURCE LINES 76-80 .. code-block:: default check = MeanAverageRecallReport().add_condition_test_average_recall_greater_than(0.4) result = check.run(test_ds, yolo) result.show(show_additional_outputs=False) .. rst-class:: sphx-glr-script-out .. code-block:: none Validating Input: | | 0/1 [Time: 00:00] Validating Input: |#####| 1/1 [Time: 00:04] Validating Input: |#####| 1/1 [Time: 00:04] Ingesting Batches: | | 0/2 [Time: 00:00] Ingesting Batches: |##5 | 1/2 [Time: 00:04] Ingesting Batches: |#####| 2/2 [Time: 00:10] Ingesting Batches: |#####| 2/2 [Time: 00:10] Computing Check: | | 0/1 [Time: 00:00] Computing Check: |#####| 1/1 [Time: 00:00] Computing Check: |#####| 1/1 [Time: 00:00] .. raw:: html
Mean Average Recall Report


.. rst-class:: sphx-glr-timing **Total running time of the script:** ( 0 minutes 31.368 seconds) .. _sphx_glr_download_checks_gallery_vision_model_evaluation_plot_mean_average_recall_report.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: plot_mean_average_recall_report.py ` .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: plot_mean_average_recall_report.ipynb ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_