Note
Go to the end to download the full example code
Confusion Matrix#
This notebooks provides an overview for using and understanding the confusion matrix check.
Structure:
What is the purpose of the check?#
The confusion matrix check outputs a confusion matrix for both classification problems and object detection problems. In object detection problems, some predictions do not overlap on any label and can be classified as not found in the confusion matrix.
Generate Dataset#
We generate a sample dataset of 128 images from the COCO dataset, and using the YOLOv5 model.
Note
In this example, we use the pytorch version of the coco dataset and model. In order to run this example using tensorflow, please change the import statements to:
from deepchecks.vision.datasets.detection import coco_tensorflow as coco
from deepchecks.vision.datasets.detection import coco_torch as coco
train_ds = coco.load_dataset(object_type='VisionData')
Downloading https://github.com/ultralytics/yolov5/releases/download/v7.0/yolov5s.pt to yolov5s.pt...
0%| | 0.00/14.1M [00:00<?, ?B/s]
100%|██████████| 14.1M/14.1M [00:00<00:00, 329MB/s]
You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
Run the check#
from deepchecks.vision.checks import ConfusionMatrixReport
check = ConfusionMatrixReport(categories_to_display=10)
result = check.run(train_ds)
result
Processing Batches:
| | 0/1 [Time: 00:00]
Processing Batches:
|█████| 1/1 [Time: 00:00]
Processing Batches:
|█████| 1/1 [Time: 00:00]
Computing Check:
| | 0/1 [Time: 00:00]
Computing Check:
|█████| 1/1 [Time: 00:00]
To display the results in an IDE like PyCharm, you can use the following code:
# result.show_in_window()
The result will be displayed in a new window.
Total running time of the script: (0 minutes 1.141 seconds)