CategoryMismatchTrainTest#

class CategoryMismatchTrainTest[source]#

Find new categories in the test set.

Parameters

columnsUnion[Hashable, List[Hashable]] , default: None: Columns to check, if none are given checks all columns except ignored ones.
ignore_columnsUnion[Hashable, List[Hashable]] , default: None: Columns to ignore, if none given checks based on columns variable.
max_features_to_showint , default: 5: maximum features with new categories to show
max_new_categories_to_showint , default: 5: maximum new categories to show in feature
aggregation_method: str, default: ‘max’: Argument for the reduce_output functionality, decides how to aggregate the vector of per-feature scores into a single aggregate score. The aggregate score value is between 0 and 1 for all methods other than l2_combination. Possible values are: ‘l2_weighted’: L2 norm over the combination of per-feature scores and feature importance, minus the L2 norm of feature importance alone, specifically, ||FI + PER_FEATURE_SCORES|| - ||FI||. This method returns a value between 0 and sqrt(n_features). ‘weighted’: Weighted mean based on feature importance, provides a robust estimation on how much the resulting score will affect the model’s performance. ‘mean’: Mean of all per-feature scores. ‘max’: Maximum of all the per-feature scores. ‘none’: No averaging. Return a dict with a per-feature score for each feature. ‘top_5’ No averaging. Return a dict with a per-feature score for top 5 features based on feature importance.
n_samplesint , default: 10_000_000: number of samples to use for this check.
random_stateint, default: 42: random seed for all check internals.

__init__(columns: Optional[Union[Hashable, List[Hashable]]] = None, ignore_columns: Optional[Union[Hashable, List[Hashable]]] = None, max_features_to_show: int = 5, max_new_categories_to_show: int = 5, aggregation_method='max', n_samples: int = 10000000, random_state: int = 42, **kwargs)[source]#

__new__(*args, **kwargs)#

Methods

`CategoryMismatchTrainTest.add_condition`(...)	Add new condition function to the check.
`CategoryMismatchTrainTest.add_condition_new_categories_less_or_equal`([...])	Add condition - require column's number of different new categories to be less or equal to threshold.
`CategoryMismatchTrainTest.add_condition_new_category_ratio_less_or_equal`([...])	Add condition - require column's ratio of instances with new categories to be less or equal to threshold.
`CategoryMismatchTrainTest.clean_conditions`()	Remove all conditions from this check instance.
`CategoryMismatchTrainTest.conditions_decision`(result)	Run conditions on given result.
`CategoryMismatchTrainTest.config`([...])	Return check configuration (conditions' configuration not yet supported).
`CategoryMismatchTrainTest.feature_reduce`(...)	Return an aggregated drift score based on aggregation method defined.
`CategoryMismatchTrainTest.from_config`(conf)	Return check object from a CheckConfig object.
`CategoryMismatchTrainTest.from_json`(conf[, ...])	Deserialize check instance from JSON string.
`CategoryMismatchTrainTest.greater_is_better`()	Return True if the check reduce_output is better when it is greater.
`CategoryMismatchTrainTest.metadata`([...])	Return check metadata.
`CategoryMismatchTrainTest.name`()	Name of class in split camel case.
`CategoryMismatchTrainTest.params`([show_defaults])	Return parameters to show when printing the check.
`CategoryMismatchTrainTest.reduce_output`(...)	Return an aggregated drift score based on aggregation method defined.
`CategoryMismatchTrainTest.remove_condition`(index)	Remove given condition by index.
`CategoryMismatchTrainTest.run`(train_dataset, ...)	Run check.
`CategoryMismatchTrainTest.run_logic`(context)	Run check.
`CategoryMismatchTrainTest.to_json`([indent, ...])	Serialize check instance to JSON string.

Examples#

train_test_validation

CategoryMismatchTrainTest.add_condition