WeakSegmentsPerformance#

class WeakSegmentsPerformance[source]#

Search for segments with low performance scores.

The check is designed to help you easily identify weak spots of your model and provide a deepdive analysis into its performance on different segments of your data. Specifically, it is designed to help you identify the model weakest segments in the data distribution for further improvement and visibility purposes.

The segments are based on the image properties - characteristics of each image such as the contrast.

In order to achieve this, the check trains several simple tree based models which try to predict the error of the user provided model on the dataset. The relevant segments are detected by analyzing the different leafs of the trained trees.

Parameters

image_propertiesList[Dict[str, Any]], default: None

List of properties. Replaces the default deepchecks properties. Each property is a dictionary with keys 'name' (str), method (Callable) and 'output_type' (str), representing attributes of said method. ‘output_type’ must be one of:

'numerical' - for continuous ordinal outputs.
'categorical' - for discrete, non-ordinal outputs. These can still be numbers, but these numbers do not have inherent value.

For more on image properties, see the guide about Vision Properties.

segment_minimum_size_ratio: float , default: 0.05

Minimum size ratio for segments. Will only search for segments of size >= segment_minimum_size_ratio * data_size.

max_categories_weak_segment: Optional[int] , default: None

Maximum number of categories that can be included in a weak segment per categorical feature. If None, the number of categories is not limited.

n_samplesOptional[int] , default: 10_000

number of samples to use for this check.

n_to_showint , default: 3

number of segments with the weakest performance to show.

categorical_aggregation_thresholdfloat , default: 0.05

For each categorical property, categories with frequency below threshold will be merged into “Other” category.

multiple_segments_per_propertybool , default: True

If True, will allow the same property to be a segmenting feature in multiple segments, otherwise each property can appear in one segment at most.

n_samplesOptional[int] , default10000

Number of samples to use for the check. If None, all samples will be used.

__init__(image_properties: Optional[List[Dict[str, Any]]] = None, n_to_show: int = 3, segment_minimum_size_ratio: float = 0.05, max_categories_weak_segment: Optional[int] = None, n_samples: Optional[int] = 10000, categorical_aggregation_threshold: float = 0.05, multiple_segments_per_property: bool = True, **kwargs)[source]#

__new__(*args, **kwargs)#

Attributes

`WeakSegmentsPerformance.categorical_aggregation_threshold`
`WeakSegmentsPerformance.max_categories_weak_segment`
`WeakSegmentsPerformance.min_category_size_ratio`
`WeakSegmentsPerformance.n_to_show`
`WeakSegmentsPerformance.n_top_features`
`WeakSegmentsPerformance.random_state`
`WeakSegmentsPerformance.segment_minimum_size_ratio`

Methods

`WeakSegmentsPerformance.add_condition`(name, ...)	Add new condition function to the check.
`WeakSegmentsPerformance.add_condition_segments_relative_performance_greater_than`([...])	Add condition - check that the score of the weakest segment is greater than supplied relative threshold.
`WeakSegmentsPerformance.clean_conditions`()	Remove all conditions from this check instance.
`WeakSegmentsPerformance.compute`(context, ...)	Find the segments with the worst performance.
`WeakSegmentsPerformance.conditions_decision`(result)	Run conditions on given result.
`WeakSegmentsPerformance.config`([...])	Return check configuration (conditions' configuration not yet supported).
`WeakSegmentsPerformance.from_config`(conf[, ...])	Return check object from a CheckConfig object.
`WeakSegmentsPerformance.from_json`(conf[, ...])	Deserialize check instance from JSON string.
`WeakSegmentsPerformance.initialize_run`(...)	Initialize the properties and sample scores states.
`WeakSegmentsPerformance.metadata`([with_doc_link])	Return check metadata.
`WeakSegmentsPerformance.name`()	Name of class in split camel case.
`WeakSegmentsPerformance.params`([show_defaults])	Return parameters to show when printing the check.
`WeakSegmentsPerformance.remove_condition`(index)	Remove given condition by index.
`WeakSegmentsPerformance.run`(dataset[, ...])	Run check.
`WeakSegmentsPerformance.to_json`([indent, ...])	Serialize check instance to JSON string.
`WeakSegmentsPerformance.update`(context, ...)	Calculate the image properties and scores per image.

Examples#

SingleDatasetPerformance.update

WeakSegmentsPerformance.categorical_aggregation_threshold