DataDuplicates

class DataDuplicates

Checks for duplicate samples in the dataset.

Parameters

columnsUnion[Hashable, List[Hashable]] , default: None

List of columns to check, if none given checks all columns Except ignored ones.

ignore_columnsUnion[Hashable, List[Hashable]] , default: None

List of columns to ignore, if none given checks based on columns variable.

n_to_showint , default: 5

number of most common duplicated samples to show.

__init__(columns: Optional[Union[Hashable, List[Hashable]]] = None, ignore_columns: Optional[Union[Hashable, List[Hashable]]] = None, n_to_show: int = 5, **kwargs)

__new__(*args, **kwargs)

Methods

DataDuplicates.add_condition(name, ...)	Add new condition function to the check.
DataDuplicates.add_condition_ratio_less_or_equal([...])	Add condition - require duplicate ratio to be less or equal to max_ratio.
DataDuplicates.clean_conditions()	Remove all conditions from this check instance.
DataDuplicates.conditions_decision(result)	Run conditions on given result.
DataDuplicates.config()	Return check configuration (conditions' configuration not yet supported).
DataDuplicates.from_config(conf)	Return check object from a CheckConfig object.
DataDuplicates.metadata([with_doc_link])	Return check metadata.
DataDuplicates.name()	Name of class in split camel case.
DataDuplicates.params([show_defaults])	Return parameters to show when printing the check.
DataDuplicates.remove_condition(index)	Remove given condition by index.
DataDuplicates.run(dataset[, model, ...])	Run check.
DataDuplicates.run_logic(context, dataset_kind)	Run check.

Examples

Data Duplicates

previous

StringLengthOutOfBounds.run_logic

next

DataDuplicates.add_condition