UnderAnnotatedMetaDataSegments#

class UnderAnnotatedMetaDataSegments[source]#

Search for under annotated data segments.

The check is designed to help you easily identify under annotated segments of your data. The segments are based on the metadata - which is data that is not part of the text, but is related to it, such as “user_id” and “user_age”. For more on metadata, see the NLP Metadata Guide.

In order to achieve this, the check trains several simple tree based models which try to predict given a sample metadata whether it will have a label. The relevant segments are detected by analyzing the different leafs of the trained trees.

Parameters
columnsUnion[Hashable, List[Hashable]] , default: None

Columns to check, if none are given checks all columns except ignored ones.

ignore_columnsUnion[Hashable, List[Hashable]] , default: None

Columns to ignore, if none given checks based on columns variable

n_top_columnsOptional[int] , default: 10

Number of features to use for segment search. Top columns are selected based on feature importance.

segment_minimum_size_ratio: float , default: 0.05

Minimum size ratio for segments. Will only search for segments of size >= segment_minimum_size_ratio * data_size.

n_samplesint , default: 10_000

Maximum number of samples to use for this check.

n_to_showint , default: 3

number of segments with the weakest performance to show.

categorical_aggregation_thresholdfloat , default: 0.05

In each categorical column, categories with frequency below threshold will be merged into “Other” category.

multiple_segments_per_columnbool , default: True

If True, will allow the same metadata column to be a segmenting column in multiple segments, otherwise each metadata column can appear in one segment at most.

__init__(columns: Optional[Union[Hashable, List[Hashable]]] = None, ignore_columns: Optional[Union[Hashable, List[Hashable]]] = None, n_top_columns: Optional[int] = 10, segment_minimum_size_ratio: float = 0.05, n_samples: int = 10000, categorical_aggregation_threshold: float = 0.05, n_to_show: int = 3, multiple_segments_per_column: bool = True, **kwargs)[source]#
__new__(*args, **kwargs)#

Attributes

UnderAnnotatedMetaDataSegments.categorical_aggregation_threshold

UnderAnnotatedMetaDataSegments.min_category_size_ratio

UnderAnnotatedMetaDataSegments.n_to_show

UnderAnnotatedMetaDataSegments.n_top_features

UnderAnnotatedMetaDataSegments.random_state

UnderAnnotatedMetaDataSegments.segment_minimum_size_ratio

Methods

UnderAnnotatedMetaDataSegments.add_condition(...)

Add new condition function to the check.

UnderAnnotatedMetaDataSegments.add_condition_segments_annotation_ratio_greater_than([...])

Add condition - check that the in all segments annotation ratio is above the provided threshold.

UnderAnnotatedMetaDataSegments.add_condition_segments_relative_performance_greater_than([...])

Add condition - check that the score of the weakest segment is greater than supplied relative threshold.

UnderAnnotatedMetaDataSegments.clean_conditions()

Remove all conditions from this check instance.

UnderAnnotatedMetaDataSegments.conditions_decision(result)

Run conditions on given result.

UnderAnnotatedMetaDataSegments.config([...])

Return check configuration (conditions' configuration not yet supported).

UnderAnnotatedMetaDataSegments.from_config(conf)

Return check object from a CheckConfig object.

UnderAnnotatedMetaDataSegments.from_json(conf)

Deserialize check instance from JSON string.

UnderAnnotatedMetaDataSegments.metadata([...])

Return check metadata.

UnderAnnotatedMetaDataSegments.name()

Name of class in split camel case.

UnderAnnotatedMetaDataSegments.params([...])

Return parameters to show when printing the check.

UnderAnnotatedMetaDataSegments.remove_condition(index)

Remove given condition by index.

UnderAnnotatedMetaDataSegments.run(dataset)

Run check.

UnderAnnotatedMetaDataSegments.run_logic(...)

Run check.

UnderAnnotatedMetaDataSegments.to_json([...])

Serialize check instance to JSON string.