SingleDatasetPerformance#

class SingleDatasetPerformance[source]#

Calculate performance metrics of a given model on a given dataset.

Parameters
scorers: Union[Dict[str, Union[Callable, str]], List[Any]] , default: None

Scorers to override the default scorers (metrics), find more about the supported formats at https://docs.deepchecks.com/stable/user-guide/general/metrics_guide.html

n_samplesOptional[int] , default10000

Number of samples to use for the check. If None, all samples will be used.

__init__(scorers: Optional[Union[Dict[str, Union[Callable, str]], List[Any]]] = None, n_samples: Optional[int] = 10000, **kwargs)[source]#
__new__(*args, **kwargs)#

Methods

SingleDatasetPerformance.add_condition(name, ...)

Add new condition function to the check.

SingleDatasetPerformance.add_condition_greater_than(...)

Add condition - the result is greater than the threshold.

SingleDatasetPerformance.add_condition_less_than(...)

Add condition - the result is less than the threshold.

SingleDatasetPerformance.clean_conditions()

Remove all conditions from this check instance.

SingleDatasetPerformance.compute(context, ...)

Compute the metric result using the ignite metrics compute method and reduce to a scalar.

SingleDatasetPerformance.conditions_decision(result)

Run conditions on given result.

SingleDatasetPerformance.config([...])

Return check configuration.

SingleDatasetPerformance.from_config(conf[, ...])

Return check object from a CheckConfig object.

SingleDatasetPerformance.from_json(conf[, ...])

Deserialize check instance from JSON string.

SingleDatasetPerformance.greater_is_better()

Return True if the check reduce_output is better when it is greater.

SingleDatasetPerformance.initialize_run(...)

Initialize the metric for the check, and validate task type is relevant.

SingleDatasetPerformance.metadata([...])

Return check metadata.

SingleDatasetPerformance.name()

Name of class in split camel case.

SingleDatasetPerformance.params([show_defaults])

Return parameters to show when printing the check.

SingleDatasetPerformance.reduce_output(...)

Return the values of the metrics for the dataset provided in a {metric: value} format.

SingleDatasetPerformance.remove_condition(index)

Remove given condition by index.

SingleDatasetPerformance.run(dataset[, ...])

Run check.

SingleDatasetPerformance.to_json([indent, ...])

Serialize check instance to JSON string.

SingleDatasetPerformance.update(context, ...)

Update the metrics by passing the batch to ignite metric update method.

Examples#