.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "checks_gallery/vision/performance/plot_mean_average_precision_report.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note Click :ref:`here ` to download the full example code .. rst-class:: sphx-glr-example-title .. _sphx_glr_checks_gallery_vision_performance_plot_mean_average_precision_report.py: Mean Average Precision Report ***************************** This notebooks provides an overview for using and understanding the mean average precision report check. **Structure:** * `What is the purpose of the check? <#what-is-the-purpose-of-the-check>`__ * `Generate data & model <#generate-data-and-model>`__ * `Run the check <#run-the-check>`__ * `Define a condition <#define-a-condition>`__ What Is the Purpose of the Check? ================================= The Mean Average Precision Report evaluates the `mAP metric `__ on the given model and data, plots the AP on graph, and returns the mAP values per bounding box size category (small, medium, large). This check only works on the Object Detection task. .. GENERATED FROM PYTHON SOURCE LINES 26-31 Generate Data and Model ================================= We generate a sample dataset of 128 images from the `COCO dataset `__, and using the `YOLOv5 model `__. .. GENERATED FROM PYTHON SOURCE LINES 31-39 .. code-block:: default from deepchecks.vision.checks.performance import MeanAveragePrecisionReport from deepchecks.vision.datasets.detection import coco yolo = coco.load_model(pretrained=True) test_ds = coco.load_dataset(train=False, object_type='VisionData') .. GENERATED FROM PYTHON SOURCE LINES 40-42 Run the check ================================= .. GENERATED FROM PYTHON SOURCE LINES 42-47 .. code-block:: default check = MeanAveragePrecisionReport() result = check.run(test_ds, yolo) result .. rst-class:: sphx-glr-script-out Out: .. code-block:: none Validating Input: 0%| | 0/1 [00:00

Mean Average Precision Report

Summarize mean average precision metrics on a dataset and model per IoU and bounding box area.

Additional Outputs
  mAP@[.50::.95] (avg.%) mAP@.50 (%) mAP@.75 (%)
Area size      
All 0.41 0.57 0.43
Small (area < 32^2) 0.21 0.34 0.21
Medium (32^2 < area < 96^2) 0.38 0.60 0.35
Large (area < 96^2) 0.54 0.67 0.59


.. GENERATED FROM PYTHON SOURCE LINES 48-53 Observe the check’s output -------------------------- The result value is a dataframe that has the Mean Average Precision score for different bounding box area sizes. We report the mAP for different IoU thresholds: 0.5, 0.75 and an average of mAP values for IoU thresholds between 0.5 and 0.9 (with a jump size of 0.05). .. GENERATED FROM PYTHON SOURCE LINES 53-56 .. code-block:: default result.value .. raw:: html
mAP@[.50::.95] (avg.%) mAP@.50 (%) mAP@.75 (%)
Area size
All 0.409436 0.566673 0.425339
Small (area < 32^2) 0.212816 0.342429 0.212868
Medium (32^2 < area < 96^2) 0.383089 0.600228 0.349863
Large (area < 96^2) 0.541146 0.674493 0.585378


.. GENERATED FROM PYTHON SOURCE LINES 57-61 Define a condition ================== We can define a condition that checks whether our model's mean average precision score is not less than a given threshold for all bounding box sizes. .. GENERATED FROM PYTHON SOURCE LINES 61-65 .. code-block:: default check = MeanAveragePrecisionReport().add_condition_average_mean_average_precision_not_less_than(0.4) result = check.run(test_ds, yolo) result.show(show_additional_outputs=False) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none Validating Input: 0%| | 0/1 [00:00 Mean Average Precision Report

.. rst-class:: sphx-glr-timing **Total running time of the script:** ( 0 minutes 35.150 seconds) .. _sphx_glr_download_checks_gallery_vision_performance_plot_mean_average_precision_report.py: .. only :: html .. container:: sphx-glr-footer :class: sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: plot_mean_average_precision_report.py ` .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: plot_mean_average_precision_report.ipynb ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_