Confusion Matrix#

This notebooks provides an overview for using and understanding the confusion matrix check.

Structure:

What is the purpose of the check?#

The confusion matrix check outputs a confusion matrix for both classification problems and object detection problems. In object detection problems, some predictions do not overlap on any label and can be classified as not found in the confusion matrix.

Generate Data and Model#

We generate a sample dataset of 128 images from the COCO dataset, and using the YOLOv5 model.

from deepchecks.vision.datasets.detection import coco

yolo = coco.load_model(pretrained=True)
train_ds = coco.load_dataset(object_type='VisionData')

Out:

Downloading https://github.com/ultralytics/yolov5/releases/download/v6.1/yolov5s.pt to yolov5s.pt...

  0%|          | 0.00/14.1M [00:00<?, ?B/s]
 43%|####2     | 6.02M/14.1M [00:00<00:00, 63.0MB/s]
100%|##########| 14.1M/14.1M [00:00<00:00, 106MB/s]

Run the check#

from deepchecks.vision.checks.performance import ConfusionMatrixReport

check = ConfusionMatrixReport(categories_to_display=10)
check.run(train_ds, yolo)

Out:

Validating Input:   0%| | 0/1 [00:00<?, ? /s]
Validating Input: 100%|#| 1/1 [00:05<00:00,  5.35s/ ]


Ingesting Batches:   0%|  | 0/2 [00:00<?, ? Batch/s]
Ingesting Batches:  50%|# | 1/2 [00:05<00:05,  5.40s/ Batch]
Ingesting Batches: 100%|##| 2/2 [00:10<00:00,  5.35s/ Batch]


Computing Check:   0%| | 0/1 [00:00<?, ? Check/s]

Confusion Matrix

Calculate the confusion matrix of the model on the given dataset.

Additional Outputs
Showing 10 of 80 classes:
"No overlapping" categories are labels and prediction which did not have a matching label/prediction.
For example a predictions that did not have a sufficiently overlapping label bounding box will appear under "No overlapping" category in the True Value axis (y-axis).


Total running time of the script: ( 0 minutes 17.120 seconds)

Gallery generated by Sphinx-Gallery