ModelErrorAnalysis#

class ModelErrorAnalysis[source]#

Find the properties that best split the data into segments of high and low model error.

Deprecated since version 0.9.3: The ModelErrorAnalysis check is deprecated and will be removed in the 0.11 version. Please use the WeakSegmentsPerformance check instead.

The check trains a regression model to predict the error of the user’s model. Then, the properties scoring the highest feature importance for the error regression model are selected and the distribution of the error vs the property values is plotted. The check results are shown only if the error regression model manages to predict the error well enough.

Parameters

image_propertiesList[Dict[str, Any]], default: None

List of properties. Replaces the default deepchecks properties. Each property is a dictionary with keys 'name' (str), method (Callable) and 'output_type' (str), representing attributes of said method. ‘output_type’ must be one of:

'numeric' - for continuous ordinal outputs.
'categorical' - for discrete, non-ordinal outputs. These can still be numbers, but these numbers do not have inherent value.

For more on image / label properties, see the guide about Data Properties.

max_properties_to_showint , default: 3

maximal number of properties to show error distribution for.

min_property_contributionfloat , default: 0.15

minimum feature importance of a property to the error regression model in order to show the property.

min_error_model_scorefloat , default: 0.5

minimum r^2 score of the error regression model for displaying the check.

min_segment_sizefloat , default: 0.05

minimal fraction of data that can comprise a weak segment.

n_display_samplesint , default: 5_000

number of samples to display in scatter plot.

random_stateint, default: 42

random seed for all check internals.

__init__(image_properties: Optional[List[Dict[str, Any]]] = None, max_properties_to_show: int = 20, min_property_contribution: float = 0.15, min_error_model_score: float = 0.5, min_segment_size: float = 0.05, n_display_samples: int = 5000, random_state: int = 42, **kwargs)[source]#

__new__(*args, **kwargs)#

Methods

`ModelErrorAnalysis.add_condition`(name, ...)	Add new condition function to the check.
`ModelErrorAnalysis.clean_conditions`()	Remove all conditions from this check instance.
`ModelErrorAnalysis.compute`(context)	Find segments that contribute to model error.
`ModelErrorAnalysis.conditions_decision`(result)	Run conditions on given result.
`ModelErrorAnalysis.config`([include_version, ...])	Return check configuration (conditions' configuration not yet supported).
`ModelErrorAnalysis.from_config`(conf[, ...])	Return check object from a CheckConfig object.
`ModelErrorAnalysis.from_json`(conf[, ...])	Deserialize check instance from JSON string.
`ModelErrorAnalysis.initialize_run`(context)	Initialize property and score lists.
`ModelErrorAnalysis.metadata`([with_doc_link])	Return check metadata.
`ModelErrorAnalysis.name`()	Name of class in split camel case.
`ModelErrorAnalysis.params`([show_defaults])	Return parameters to show when printing the check.
`ModelErrorAnalysis.remove_condition`(index)	Remove given condition by index.
`ModelErrorAnalysis.run`(train_dataset, ...[, ...])	Run check.
`ModelErrorAnalysis.to_json`([indent, ...])	Serialize check instance to JSON string.
`ModelErrorAnalysis.update`(context, batch, ...)	Accumulate property data of images and scores.

Examples#

MeanAverageRecallReport.update

ModelErrorAnalysis.add_condition