Export Outputs to JSON#

In this guide, we will demonstrate how to export the Check’s and the Suite’s output to JSON format.

This enables to use the exported results as a serializeable object that can later be shared or reviewed in other Python scopes, and also enables working with the check’s output (name, parameters, result value and result display) not only in a visual manner (e.g. by parsing the output JSON).

Structure:

Load Data#

from deepchecks.tabular.datasets.classification import iris

train_dataset, test_dataset = iris.load_data()
model = iris.load_fitted_model()

Run a Check#

from deepchecks.tabular.checks import WholeDatasetDrift

result = WholeDatasetDrift().add_condition_overall_drift_value_less_than(
).run(train_dataset, test_dataset, model)

Observe CheckResult Display and Value

result
Whole Dataset Drift


result.value
{'domain_classifier_auc': 0.4545454545454546, 'domain_classifier_drift_score': 0, 'domain_classifier_feature_importance': {'petal length (cm)': 1.0, 'sepal length (cm)': 0.0, 'sepal width (cm)': 0.0, 'petal width (cm)': 0.0}}

Export a Check’s Output (CheckResult) to a JSON Format#

Serialization of the output to a JSON format is possible using the to_json function. This function takes the check outputs and serializes it to a JSON string.

The format of the check’s output json includes all info required to reconstruct the check run and it’s output: it’s name, the parameters the check receives, one sentence summary of the check’s purpose, it’s result value and a json of the data relevant for displaying the check’s outputs.

See Check JSON Structure#

from deepchecks.core import CheckResult

help(CheckResult.to_json)
Help on function to_json in module deepchecks.core.check_result:

to_json(self, with_display: bool = True, **kwargs) -> str
    Serialize result into a json string.

    Returned JSON string will have next structure:

    >>    class CheckResultMetadata(TypedDict):
    >>        type: str
    >>        check: CheckMetadata
    >>        value: Any
    >>        header: str
    >>        conditions_results: List[Dict[Any, Any]]
    >>        display: List[Dict[str, Any]]

    >>    class CheckMetadata(TypedDict):
    >>        name: str
    >>        params: Dict[Any, Any]
    >>        summary: str

    Parameters
    ----------
    with_display : bool
        whethere to include display items or not

    Returns
    -------
    str
# get output JSON
serialized_output = result.to_json()
import json

# note - conditions_table key exists only if there are conditions that were defined on check
json.loads(serialized_output).keys()
dict_keys(['type', 'check', 'header', 'value', 'conditions_results', 'display'])

Observe JSON Output#

json.loads(serialized_output)
{'type': 'CheckResult', 'check': {'name': 'Whole Dataset Drift', 'params': {'n_top_columns': 3, 'min_feature_importance': 0.05, 'max_num_categories_for_display': 10, 'show_categories_by': 'largest_difference', 'sample_size': 10000, 'random_state': 42, 'test_size': 0.3, 'min_meaningful_drift_score': 0.05, 'max_num_categories': None}, 'summary': '    Calculate drift between the entire train and test datasets using a model trained to distinguish between them. <a href="https://docs.deepchecks.com/stable/checks_gallery/tabular/train_test_validation/plot_whole_dataset_drift.html?utm_source=display_output&utm_medium=referral&utm_campaign=check_link" target="_blank">Read More...</a>'}, 'header': 'Whole Dataset Drift', 'value': {'domain_classifier_auc': {'dtype': 'float64', 'value': 0.4545454545454546}, 'domain_classifier_drift_score': 0, 'domain_classifier_feature_importance': {'petal length (cm)': 1.0, 'sepal length (cm)': 0.0, 'sepal width (cm)': 0.0, 'petal width (cm)': 0.0}}, 'conditions_results': [{'Status': 'PASS', 'Condition': 'Drift value is less than 0.25', 'More Info': 'Found drift value of: 0, corresponding to a domain classifier AUC of: 0.45'}], 'display': []}

Display the CheckResult Output from a JSON#

The serialized JSON can be used to reproduce the run in other Python scopes, by using the from_json function

from deepchecks.utils.json_utils import from_json

from_json(serialized_output)
Whole Dataset Drift


Exporting a Suite’s Output (SuiteResult) to JSON#

Run Suite and Save to JSON#

from deepchecks.tabular.suites import full_suite

suite = full_suite()
suite_result = suite.run(train_dataset=train_dataset, test_dataset=test_dataset, model=model)
suite_json = suite_result.to_json()
Full Suite:
|                                    | 0/36 [Time: 00:00]
Full Suite:
|#                                   | 1/36 [Time: 00:00, Check=Train Test Performance]
Full Suite:
|####                                | 4/36 [Time: 00:00, Check=Train Test Prediction Drift]
Full Suite:
|######                              | 6/36 [Time: 00:01, Check=Weak Segments Performance]
Full Suite:
|####################                | 20/36 [Time: 00:01, Check=Train Test Samples Mix]
Full Suite:
|##########################          | 26/36 [Time: 00:02, Check=Special Characters]
Full Suite:
|##################################  | 34/36 [Time: 00:02, Check=Feature Label Correlation]

Observe Suite’s JSON Strucutre#

from deepchecks.core import SuiteResult

help(SuiteResult.to_json)
Help on function to_json in module deepchecks.core.suite:

to_json(self, with_display: bool = True, **kwargs)
    Return check result as json.

    Parameters
    ----------
    with_display : bool, default True
        whether to include serialized `SuiteResult.display` items into
        the output or not

    Returns
    -------
    str

Suite name:

json.loads(suite_json)['name']
'Full Suite'

Results is an array of CheckResult JSON’s, let’s see how many checks ran in suite:

len(json.loads(suite_json)['results'])
55

Lets observe a specific check’s output, which is similar to the Check output’s JSON Structure we saw before for each check

json.loads(suite_json)['results'][0]
{'type': 'CheckResult', 'check': {'name': 'Train Test Performance', 'params': {'scorers': None, 'reduce': 'mean'}, 'summary': 'Summarize given model performance on the train and test datasets based on selected scorers. <a href="https://docs.deepchecks.com/stable/checks_gallery/tabular/model_evaluation/plot_train_test_performance.html?utm_source=display_output&utm_medium=referral&utm_campaign=check_link" target="_blank">Read More...</a>'}, 'header': 'Train Test Performance', 'value': '[{"Dataset":"Train","Class":0,"Metric":"F1","Value":1.0,"Number of samples":37},{"Dataset":"Train","Class":1,"Metric":"F1","Value":1.0,"Number of samples":37},{"Dataset":"Train","Class":2,"Metric":"F1","Value":1.0,"Number of samples":38},{"Dataset":"Train","Class":0,"Metric":"Precision","Value":1.0,"Number of samples":37},{"Dataset":"Train","Class":1,"Metric":"Precision","Value":1.0,"Number of samples":37},{"Dataset":"Train","Class":2,"Metric":"Precision","Value":1.0,"Number of samples":38},{"Dataset":"Train","Class":0,"Metric":"Recall","Value":1.0,"Number of samples":37},{"Dataset":"Train","Class":1,"Metric":"Recall","Value":1.0,"Number of samples":37},{"Dataset":"Train","Class":2,"Metric":"Recall","Value":1.0,"Number of samples":38},{"Dataset":"Test","Class":0,"Metric":"F1","Value":1.0,"Number of samples":13},{"Dataset":"Test","Class":1,"Metric":"F1","Value":0.9285714286,"Number of samples":13},{"Dataset":"Test","Class":2,"Metric":"F1","Value":0.9090909091,"Number of samples":12},{"Dataset":"Test","Class":0,"Metric":"Precision","Value":1.0,"Number of samples":13},{"Dataset":"Test","Class":1,"Metric":"Precision","Value":0.8666666667,"Number of samples":13},{"Dataset":"Test","Class":2,"Metric":"Precision","Value":1.0,"Number of samples":12},{"Dataset":"Test","Class":0,"Metric":"Recall","Value":1.0,"Number of samples":13},{"Dataset":"Test","Class":1,"Metric":"Recall","Value":1.0,"Number of samples":13},{"Dataset":"Test","Class":2,"Metric":"Recall","Value":0.8333333333,"Number of samples":12}]', 'conditions_results': [{'Status': 'FAIL', 'Condition': 'Train-Test scores relative degradation is less than 0.1', 'More Info': '2 scores failed. Found max degradation of 16.67% for metric Recall and class 2.'}], 'display': [{'type': 'plotly', 'payload': '{"data":[{"alignmentgroup":"True","bingroup":"x","histfunc":"sum","hovertemplate":"Dataset=Train<br>Metric=F1<br>Class=%{x}<br>sum of Value=%{y}<extra></extra>","legendgroup":"Train","marker":{"color":"#00008b","pattern":{"shape":""}},"name":"Train","offsetgroup":"Train","orientation":"v","showlegend":true,"x":[0,1,2],"xaxis":"x","y":[1.0,1.0,1.0],"yaxis":"y","type":"histogram"},{"alignmentgroup":"True","bingroup":"x","histfunc":"sum","hovertemplate":"Dataset=Train<br>Metric=Precision<br>Class=%{x}<br>sum of Value=%{y}<extra></extra>","legendgroup":"Train","marker":{"color":"#00008b","pattern":{"shape":""}},"name":"Train","offsetgroup":"Train","orientation":"v","showlegend":false,"x":[0,1,2],"xaxis":"x2","y":[1.0,1.0,1.0],"yaxis":"y2","type":"histogram"},{"alignmentgroup":"True","bingroup":"x","histfunc":"sum","hovertemplate":"Dataset=Train<br>Metric=Recall<br>Class=%{x}<br>sum of Value=%{y}<extra></extra>","legendgroup":"Train","marker":{"color":"#00008b","pattern":{"shape":""}},"name":"Train","offsetgroup":"Train","orientation":"v","showlegend":false,"x":[0,1,2],"xaxis":"x3","y":[1.0,1.0,1.0],"yaxis":"y3","type":"histogram"},{"alignmentgroup":"True","bingroup":"x","histfunc":"sum","hovertemplate":"Dataset=Test<br>Metric=F1<br>Class=%{x}<br>sum of Value=%{y}<extra></extra>","legendgroup":"Test","marker":{"color":"#69b3a2","pattern":{"shape":""}},"name":"Test","offsetgroup":"Test","orientation":"v","showlegend":true,"x":[0,1,2],"xaxis":"x","y":[1.0,0.9285714285714286,0.9090909090909091],"yaxis":"y","type":"histogram"},{"alignmentgroup":"True","bingroup":"x","histfunc":"sum","hovertemplate":"Dataset=Test<br>Metric=Precision<br>Class=%{x}<br>sum of Value=%{y}<extra></extra>","legendgroup":"Test","marker":{"color":"#69b3a2","pattern":{"shape":""}},"name":"Test","offsetgroup":"Test","orientation":"v","showlegend":false,"x":[0,1,2],"xaxis":"x2","y":[1.0,0.8666666666666667,1.0],"yaxis":"y2","type":"histogram"},{"alignmentgroup":"True","bingroup":"x","histfunc":"sum","hovertemplate":"Dataset=Test<br>Metric=Recall<br>Class=%{x}<br>sum of Value=%{y}<extra></extra>","legendgroup":"Test","marker":{"color":"#69b3a2","pattern":{"shape":""}},"name":"Test","offsetgroup":"Test","orientation":"v","showlegend":false,"x":[0,1,2],"xaxis":"x3","y":[1.0,1.0,0.8333333333333334],"yaxis":"y3","type":"histogram"}],"layout":{"template":{"data":{"bar":[{"error_x":{"color":"#2a3f5f"},"error_y":{"color":"#2a3f5f"},"marker":{"line":{"color":"#E5ECF6","width":0.5},"pattern":{"fillmode":"overlay","size":10,"solidity":0.2}},"type":"bar"}],"barpolar":[{"marker":{"line":{"color":"#E5ECF6","width":0.5},"pattern":{"fillmode":"overlay","size":10,"solidity":0.2}},"type":"barpolar"}],"carpet":[{"aaxis":{"endlinecolor":"#2a3f5f","gridcolor":"white","linecolor":"white","minorgridcolor":"white","startlinecolor":"#2a3f5f"},"baxis":{"endlinecolor":"#2a3f5f","gridcolor":"white","linecolor":"white","minorgridcolor":"white","startlinecolor":"#2a3f5f"},"type":"carpet"}],"choropleth":[{"colorbar":{"outlinewidth":0,"ticks":""},"type":"choropleth"}],"contour":[{"colorbar":{"outlinewidth":0,"ticks":""},"colorscale":[[0.0,"#0d0887"],[0.1111111111111111,"#46039f"],[0.2222222222222222,"#7201a8"],[0.3333333333333333,"#9c179e"],[0.4444444444444444,"#bd3786"],[0.5555555555555556,"#d8576b"],[0.6666666666666666,"#ed7953"],[0.7777777777777778,"#fb9f3a"],[0.8888888888888888,"#fdca26"],[1.0,"#f0f921"]],"type":"contour"}],"contourcarpet":[{"colorbar":{"outlinewidth":0,"ticks":""},"type":"contourcarpet"}],"heatmap":[{"colorbar":{"outlinewidth":0,"ticks":""},"colorscale":[[0.0,"#0d0887"],[0.1111111111111111,"#46039f"],[0.2222222222222222,"#7201a8"],[0.3333333333333333,"#9c179e"],[0.4444444444444444,"#bd3786"],[0.5555555555555556,"#d8576b"],[0.6666666666666666,"#ed7953"],[0.7777777777777778,"#fb9f3a"],[0.8888888888888888,"#fdca26"],[1.0,"#f0f921"]],"type":"heatmap"}],"heatmapgl":[{"colorbar":{"outlinewidth":0,"ticks":""},"colorscale":[[0.0,"#0d0887"],[0.1111111111111111,"#46039f"],[0.2222222222222222,"#7201a8"],[0.3333333333333333,"#9c179e"],[0.4444444444444444,"#bd3786"],[0.5555555555555556,"#d8576b"],[0.6666666666666666,"#ed7953"],[0.7777777777777778,"#fb9f3a"],[0.8888888888888888,"#fdca26"],[1.0,"#f0f921"]],"type":"heatmapgl"}],"histogram":[{"marker":{"pattern":{"fillmode":"overlay","size":10,"solidity":0.2}},"type":"histogram"}],"histogram2d":[{"colorbar":{"outlinewidth":0,"ticks":""},"colorscale":[[0.0,"#0d0887"],[0.1111111111111111,"#46039f"],[0.2222222222222222,"#7201a8"],[0.3333333333333333,"#9c179e"],[0.4444444444444444,"#bd3786"],[0.5555555555555556,"#d8576b"],[0.6666666666666666,"#ed7953"],[0.7777777777777778,"#fb9f3a"],[0.8888888888888888,"#fdca26"],[1.0,"#f0f921"]],"type":"histogram2d"}],"histogram2dcontour":[{"colorbar":{"outlinewidth":0,"ticks":""},"colorscale":[[0.0,"#0d0887"],[0.1111111111111111,"#46039f"],[0.2222222222222222,"#7201a8"],[0.3333333333333333,"#9c179e"],[0.4444444444444444,"#bd3786"],[0.5555555555555556,"#d8576b"],[0.6666666666666666,"#ed7953"],[0.7777777777777778,"#fb9f3a"],[0.8888888888888888,"#fdca26"],[1.0,"#f0f921"]],"type":"histogram2dcontour"}],"mesh3d":[{"colorbar":{"outlinewidth":0,"ticks":""},"type":"mesh3d"}],"parcoords":[{"line":{"colorbar":{"outlinewidth":0,"ticks":""}},"type":"parcoords"}],"pie":[{"automargin":true,"type":"pie"}],"scatter":[{"marker":{"colorbar":{"outlinewidth":0,"ticks":""}},"type":"scatter"}],"scatter3d":[{"line":{"colorbar":{"outlinewidth":0,"ticks":""}},"marker":{"colorbar":{"outlinewidth":0,"ticks":""}},"type":"scatter3d"}],"scattercarpet":[{"marker":{"colorbar":{"outlinewidth":0,"ticks":""}},"type":"scattercarpet"}],"scattergeo":[{"marker":{"colorbar":{"outlinewidth":0,"ticks":""}},"type":"scattergeo"}],"scattergl":[{"marker":{"colorbar":{"outlinewidth":0,"ticks":""}},"type":"scattergl"}],"scattermapbox":[{"marker":{"colorbar":{"outlinewidth":0,"ticks":""}},"type":"scattermapbox"}],"scatterpolar":[{"marker":{"colorbar":{"outlinewidth":0,"ticks":""}},"type":"scatterpolar"}],"scatterpolargl":[{"marker":{"colorbar":{"outlinewidth":0,"ticks":""}},"type":"scatterpolargl"}],"scatterternary":[{"marker":{"colorbar":{"outlinewidth":0,"ticks":""}},"type":"scatterternary"}],"surface":[{"colorbar":{"outlinewidth":0,"ticks":""},"colorscale":[[0.0,"#0d0887"],[0.1111111111111111,"#46039f"],[0.2222222222222222,"#7201a8"],[0.3333333333333333,"#9c179e"],[0.4444444444444444,"#bd3786"],[0.5555555555555556,"#d8576b"],[0.6666666666666666,"#ed7953"],[0.7777777777777778,"#fb9f3a"],[0.8888888888888888,"#fdca26"],[1.0,"#f0f921"]],"type":"surface"}],"table":[{"cells":{"fill":{"color":"#EBF0F8"},"line":{"color":"white"}},"header":{"fill":{"color":"#C8D4E3"},"line":{"color":"white"}},"type":"table"}]},"layout":{"annotationdefaults":{"arrowcolor":"#2a3f5f","arrowhead":0,"arrowwidth":1},"autotypenumbers":"strict","coloraxis":{"colorbar":{"outlinewidth":0,"ticks":""}},"colorscale":{"diverging":[[0,"#8e0152"],[0.1,"#c51b7d"],[0.2,"#de77ae"],[0.3,"#f1b6da"],[0.4,"#fde0ef"],[0.5,"#f7f7f7"],[0.6,"#e6f5d0"],[0.7,"#b8e186"],[0.8,"#7fbc41"],[0.9,"#4d9221"],[1,"#276419"]],"sequential":[[0.0,"#0d0887"],[0.1111111111111111,"#46039f"],[0.2222222222222222,"#7201a8"],[0.3333333333333333,"#9c179e"],[0.4444444444444444,"#bd3786"],[0.5555555555555556,"#d8576b"],[0.6666666666666666,"#ed7953"],[0.7777777777777778,"#fb9f3a"],[0.8888888888888888,"#fdca26"],[1.0,"#f0f921"]],"sequentialminus":[[0.0,"#0d0887"],[0.1111111111111111,"#46039f"],[0.2222222222222222,"#7201a8"],[0.3333333333333333,"#9c179e"],[0.4444444444444444,"#bd3786"],[0.5555555555555556,"#d8576b"],[0.6666666666666666,"#ed7953"],[0.7777777777777778,"#fb9f3a"],[0.8888888888888888,"#fdca26"],[1.0,"#f0f921"]]},"colorway":["#636efa","#EF553B","#00cc96","#ab63fa","#FFA15A","#19d3f3","#FF6692","#B6E880","#FF97FF","#FECB52"],"font":{"color":"#2a3f5f"},"geo":{"bgcolor":"white","lakecolor":"white","landcolor":"#E5ECF6","showlakes":true,"showland":true,"subunitcolor":"white"},"hoverlabel":{"align":"left"},"hovermode":"closest","mapbox":{"style":"light"},"paper_bgcolor":"white","plot_bgcolor":"#E5ECF6","polar":{"angularaxis":{"gridcolor":"white","linecolor":"white","ticks":""},"bgcolor":"#E5ECF6","radialaxis":{"gridcolor":"white","linecolor":"white","ticks":""}},"scene":{"xaxis":{"backgroundcolor":"#E5ECF6","gridcolor":"white","gridwidth":2,"linecolor":"white","showbackground":true,"ticks":"","zerolinecolor":"white"},"yaxis":{"backgroundcolor":"#E5ECF6","gridcolor":"white","gridwidth":2,"linecolor":"white","showbackground":true,"ticks":"","zerolinecolor":"white"},"zaxis":{"backgroundcolor":"#E5ECF6","gridcolor":"white","gridwidth":2,"linecolor":"white","showbackground":true,"ticks":"","zerolinecolor":"white"}},"shapedefaults":{"line":{"color":"#2a3f5f"}},"ternary":{"aaxis":{"gridcolor":"white","linecolor":"white","ticks":""},"baxis":{"gridcolor":"white","linecolor":"white","ticks":""},"bgcolor":"#E5ECF6","caxis":{"gridcolor":"white","linecolor":"white","ticks":""}},"title":{"x":0.05},"xaxis":{"automargin":true,"gridcolor":"white","linecolor":"white","ticks":"","title":{"standoff":15},"zerolinecolor":"white","zerolinewidth":2},"yaxis":{"automargin":true,"gridcolor":"white","linecolor":"white","ticks":"","title":{"standoff":15},"zerolinecolor":"white","zerolinewidth":2}}},"xaxis":{"anchor":"y","domain":[0.0,0.3],"title":{},"tickprefix":"Class ","tickangle":60,"type":"category"},"yaxis":{"anchor":"x","domain":[0.0,1.0],"title":{},"showticklabels":true},"xaxis2":{"anchor":"y2","domain":[0.35,0.6499999999999999],"matches":"x","title":{},"tickprefix":"Class ","tickangle":60,"type":"category"},"yaxis2":{"anchor":"x2","domain":[0.0,1.0],"showticklabels":true,"title":{}},"xaxis3":{"anchor":"y3","domain":[0.7,1.0],"matches":"x","title":{},"tickprefix":"Class ","tickangle":60,"type":"category"},"yaxis3":{"anchor":"x3","domain":[0.0,1.0],"showticklabels":true,"title":{}},"annotations":[{"font":{},"showarrow":false,"text":"F1","x":0.15,"xanchor":"center","xref":"paper","y":1.0,"yanchor":"bottom","yref":"paper"},{"font":{},"showarrow":false,"text":"Precision","x":0.49999999999999994,"xanchor":"center","xref":"paper","y":1.0,"yanchor":"bottom","yref":"paper"},{"font":{},"showarrow":false,"text":"Recall","x":0.85,"xanchor":"center","xref":"paper","y":1.0,"yanchor":"bottom","yref":"paper"}],"legend":{"title":{"text":"Dataset"},"tracegroupgap":0},"margin":{"t":60},"barmode":"group"}}'}]}

Total running time of the script: ( 0 minutes 4.919 seconds)

Gallery generated by Sphinx-Gallery