.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "checks_gallery/vision/performance/plot_mean_average_recall_report.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note Click :ref:`here ` to download the full example code .. rst-class:: sphx-glr-example-title .. _sphx_glr_checks_gallery_vision_performance_plot_mean_average_recall_report.py: Mean Average Recall Report ************************** This notebooks provides an overview for using and understanding the mean average recall report check. **Structure:** * `What is the purpose of the check? <#what-is-the-purpose-of-the-check>`__ * `Generate data & model <#generate-data-and-model>`__ * `Run the check <#run-the-check>`__ * `Define a condition <#define-a-condition>`__ What is the purpose of the check? ================================= The Mean Average Recall Report evaluates the `mAR metric `__ on the given model and data, and returns the mAR values per bounding box size category (small, medium, large). This check only works on the Object Detection task. .. GENERATED FROM PYTHON SOURCE LINES 24-26 Imports ------- .. GENERATED FROM PYTHON SOURCE LINES 26-32 .. code-block:: default import numpy as np from deepchecks.vision.checks.performance import MeanAverageRecallReport from deepchecks.vision.datasets.detection import coco .. GENERATED FROM PYTHON SOURCE LINES 33-40 Generate Data and Model ----------------------- We generate a sample dataset of 128 images from the `COCO dataset `__, and using the `YOLOv5 model `__. For the label formatter - our dataset returns exactly the accepted format, so our formatting function is the simple `lambda x: x` function. .. GENERATED FROM PYTHON SOURCE LINES 40-45 .. code-block:: default yolo = coco.load_model(pretrained=True) test_ds = coco.load_dataset(train=False, object_type='VisionData') .. GENERATED FROM PYTHON SOURCE LINES 46-48 Run the check ------------- .. GENERATED FROM PYTHON SOURCE LINES 48-53 .. code-block:: default check = MeanAverageRecallReport() result = check.run(test_ds, yolo) result .. rst-class:: sphx-glr-script-out Out: .. code-block:: none Validating Input: 0%| | 0/1 [00:00

Mean Average Recall Report

Summarize mean average recall metrics on a dataset and model per detections and area range.

Additional Outputs
  AR@1 (%) AR@10 (%) AR@100 (%)
Area size      
All 0.33 0.42 0.43
Small (area < 32^2) 0.10 0.22 0.22
Medium (32^2 < area < 96^2) 0.33 0.42 0.42
Large (area < 96^2) 0.48 0.54 0.55


.. GENERATED FROM PYTHON SOURCE LINES 54-57 Observe the check’s output -------------------------- The result value is a dataframe that has the average recall score per each area range and IoU. .. GENERATED FROM PYTHON SOURCE LINES 57-60 .. code-block:: default result.value .. raw:: html
AR@1 (%) AR@10 (%) AR@100 (%)
Area size
All 0.330552 0.423444 0.429179
Small (area < 32^2) 0.104955 0.220594 0.220594
Medium (32^2 < area < 96^2) 0.325099 0.417392 0.423844
Large (area < 96^2) 0.481611 0.544408 0.549963


.. GENERATED FROM PYTHON SOURCE LINES 61-65 Define a condition ================== We can define a condition that checks whether our model's average recall score is not less than a given threshold .. GENERATED FROM PYTHON SOURCE LINES 65-69 .. code-block:: default check = MeanAverageRecallReport().add_condition_test_average_recall_not_less_than(0.4) result = check.run(test_ds, yolo) result.show(show_additional_outputs=False) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none Validating Input: 0%| | 0/1 [00:00 Mean Average Recall Report

.. rst-class:: sphx-glr-timing **Total running time of the script:** ( 0 minutes 35.458 seconds) .. _sphx_glr_download_checks_gallery_vision_performance_plot_mean_average_recall_report.py: .. only :: html .. container:: sphx-glr-footer :class: sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: plot_mean_average_recall_report.py ` .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: plot_mean_average_recall_report.ipynb ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_