CategoryMismatchTrainTest#

class CategoryMismatchTrainTest[source]#

Find new categories in the test set.

Parameters
columnsUnion[Hashable, List[Hashable]] , default: None

Columns to check, if none are given checks all columns except ignored ones.

ignore_columnsUnion[Hashable, List[Hashable]] , default: None

Columns to ignore, if none given checks based on columns variable.

max_features_to_showint , default: 5

maximum features with new categories to show

max_new_categories_to_showint , default: 5

maximum new categories to show in feature

aggregation_method: str, default: ‘max’

argument for the reduce_output functionality, decides how to aggregate the drift scores for a collective score. The collective score value is between 0 and 1 for all methods other than l2_combination. Possible values are: ‘l2_weighted’: L2 norm over the combination of drift scores and feature importance, minus the L2 norm of feature importance alone, specifically, ||FI + DRIFT|| - ||FI||. This method returns a value between 0 and sqrt(n_features). ‘weighted’: Weighted mean based on feature importance, provides a robust estimation on how much the drift will affect the model’s performance. ‘mean’: Mean of all drift scores. ‘max’: Maximum of all the features drift scores. ‘none’: No averaging. Return a dict with a drift score for each feature. ‘top_5’ No averaging. Return a dict with a drift score for top 5 features based on feature importance.

n_samplesint , default: 10_000_000

number of samples to use for this check.

random_stateint, default: 42

random seed for all check internals.

__init__(columns: Optional[Union[Hashable, List[Hashable]]] = None, ignore_columns: Optional[Union[Hashable, List[Hashable]]] = None, max_features_to_show: int = 5, max_new_categories_to_show: int = 5, aggregation_method='max', n_samples: int = 10000000, random_state: int = 42, **kwargs)[source]#
__new__(*args, **kwargs)#

Methods

CategoryMismatchTrainTest.add_condition(...)

Add new condition function to the check.

CategoryMismatchTrainTest.add_condition_new_categories_less_or_equal([...])

Add condition - require column's number of different new categories to be less or equal to threshold.

CategoryMismatchTrainTest.add_condition_new_category_ratio_less_or_equal([...])

Add condition - require column's ratio of instances with new categories to be less or equal to threshold.

CategoryMismatchTrainTest.clean_conditions()

Remove all conditions from this check instance.

CategoryMismatchTrainTest.conditions_decision(result)

Run conditions on given result.

CategoryMismatchTrainTest.config([...])

Return check configuration (conditions' configuration not yet supported).

CategoryMismatchTrainTest.feature_reduce(...)

Return an aggregated drift score based on aggregation method defined.

CategoryMismatchTrainTest.from_config(conf)

Return check object from a CheckConfig object.

CategoryMismatchTrainTest.from_json(conf[, ...])

Deserialize check instance from JSON string.

CategoryMismatchTrainTest.metadata([...])

Return check metadata.

CategoryMismatchTrainTest.name()

Name of class in split camel case.

CategoryMismatchTrainTest.params([show_defaults])

Return parameters to show when printing the check.

CategoryMismatchTrainTest.reduce_output(...)

Return an aggregated drift score based on aggregation method defined.

CategoryMismatchTrainTest.remove_condition(index)

Remove given condition by index.

CategoryMismatchTrainTest.run(train_dataset, ...)

Run check.

CategoryMismatchTrainTest.run_logic(context)

Run check.

CategoryMismatchTrainTest.to_json([indent])

Serialize check instance to JSON string.

Examples#