UnderAnnotatedMetaDataSegments#
- class UnderAnnotatedMetaDataSegments[source]#
Search for under annotated data segments.
The check is designed to help you easily identify under annotated segments of your data. The segments are based on the metadata - which is data that is not part of the text, but is related to it, such as “user_id” and “user_age”. For more on metadata, see the NLP Metadata Guide.
In order to achieve this, the check trains several simple tree based models which try to predict given a sample metadata whether it will have a label. The relevant segments are detected by analyzing the different leafs of the trained trees.
- Parameters
- columnsUnion[Hashable, List[Hashable]] , default: None
Columns to check, if none are given checks all columns except ignored ones.
- ignore_columnsUnion[Hashable, List[Hashable]] , default: None
Columns to ignore, if none given checks based on columns variable
- n_top_columnsOptional[int] , default: 10
Number of features to use for segment search. Top columns are selected based on feature importance.
- segment_minimum_size_ratio: float , default: 0.05
Minimum size ratio for segments. Will only search for segments of size >= segment_minimum_size_ratio * data_size.
- max_categories_weak_segment: Optional[int] , default: None
Maximum number of categories that can be included in a weak segment per categorical feature. If None, the number of categories is not limited.
- n_samplesint , default: 10_000
Maximum number of samples to use for this check.
- n_to_showint , default: 3
number of segments with the weakest performance to show.
- categorical_aggregation_thresholdfloat , default: 0.05
In each categorical column, categories with frequency below threshold will be merged into “Other” category.
- multiple_segments_per_columnbool , default: True
If True, will allow the same metadata column to be a segmenting column in multiple segments, otherwise each metadata column can appear in one segment at most.
- __init__(columns: Optional[Union[Hashable, List[Hashable]]] = None, ignore_columns: Optional[Union[Hashable, List[Hashable]]] = None, n_top_columns: Optional[int] = 10, segment_minimum_size_ratio: float = 0.05, max_categories_weak_segment: Optional[int] = None, n_samples: int = 10000, categorical_aggregation_threshold: float = 0.05, n_to_show: int = 3, multiple_segments_per_column: bool = True, **kwargs)[source]#
- __new__(*args, **kwargs)#
Attributes
|
|
Methods
Add new condition function to the check. |
|
|
Add condition - check that the in all segments annotation ratio is above the provided threshold. |
|
Add condition - check that the score of the weakest segment is greater than supplied relative threshold. |
Remove all conditions from this check instance. |
|
Run conditions on given result. |
|
Return check configuration (conditions' configuration not yet supported). |
|
Return check object from a CheckConfig object. |
|
Deserialize check instance from JSON string. |
|
Return check metadata. |
|
Name of class in split camel case. |
|
Return parameters to show when printing the check. |
|
Remove given condition by index. |
|
|
Run check. |
Run check. |
|
Serialize check instance to JSON string. |