Note
Click here to download the full example code
Confusion Matrix#
This notebooks provides an overview for using and understanding the confusion matrix check.
Structure:
What is the purpose of the check?#
The confusion matrix check outputs a confusion matrix for both classification problems and object detection problems. In object detection problems, some predictions do not overlap on any label and can be classified as not found in the confusion matrix.
Generate Data and Model#
We generate a sample dataset of 128 images from the COCO dataset, and using the YOLOv5 model.
from deepchecks.vision.datasets.detection import coco
yolo = coco.load_model(pretrained=True)
train_ds = coco.load_dataset(object_type='VisionData')
Downloading: "https://github.com/ultralytics/yolov5/archive/v6.1.zip" to /home/runner/.cache/torch/hub/v6.1.zip
Downloading https://github.com/ultralytics/yolov5/releases/download/v6.2/yolov5s.pt to yolov5s.pt...
0%| | 0.00/14.1M [00:00<?, ?B/s]
40%|###9 | 5.59M/14.1M [00:00<00:00, 57.4MB/s]
78%|#######8 | 11.1M/14.1M [00:00<00:00, 52.8MB/s]
100%|##########| 14.1M/14.1M [00:00<00:00, 50.2MB/s]
Run the check#
from deepchecks.vision.checks import ConfusionMatrixReport
check = ConfusionMatrixReport(categories_to_display=10)
result = check.run(train_ds, yolo)
result
Validating Input:
| | 0/1 [Time: 00:00]
Validating Input:
|#####| 1/1 [Time: 00:05]
Validating Input:
|#####| 1/1 [Time: 00:05]
Ingesting Batches:
| | 0/2 [Time: 00:00]
Ingesting Batches:
|##5 | 1/2 [Time: 00:05]
Ingesting Batches:
|#####| 2/2 [Time: 00:10]
Ingesting Batches:
|#####| 2/2 [Time: 00:10]
Computing Check:
| | 0/1 [Time: 00:00]
Computing Check:
|#####| 1/1 [Time: 00:00]
If you have a GPU, you can speed up this check by calling:
# check.run(train_ds, yolo, device=<your GPU>)
To display the results in an IDE like PyCharm, you can use the following code:
# result.show_in_window()
The result will be displayed in a new window.
Total running time of the script: ( 0 minutes 19.628 seconds)