model_evaluation#

Module containing the model evaluation checks in the vision package.

Classes

TrainTestPredictionDrift

Calculate prediction drift between train dataset and test dataset, using statistical measures.

ClassPerformance

Summarize given metrics on a dataset and model.

ConfusionMatrixReport

Calculate the confusion matrix of the model on the given dataset.

ImageSegmentPerformance

Segment the data by various properties of the image, and compare the performance of the segments.

MeanAveragePrecisionReport

Summarize mean average precision metrics on a dataset and model per IoU and bounding box area.

MeanAverageRecallReport

Summarize mean average recall metrics on a dataset and model per detections and area range.

ModelErrorAnalysis

Find the properties that best split the data into segments of high and low model error.

RobustnessReport

Compare performance of model on original dataset and augmented dataset.

SimpleModelComparison

Compare given model score to simple model score (according to given model type).

SingleDatasetPerformance

Calculate performance metrics of a given model on a given dataset.