CML#

CML is a CLI from from the creators of DVC - Iterative AI - that helps integrate your machine learning projects in your CI pipeline.

Deepchecks has an option to save the results of a suite as a summary markdown that includes the full html report as an attachment - as GitHub markdown and GitLab markdown do not run javascript.

The example here is written for GitLab CI, but the same principles apply in other CI systems.

Export SuiteResult as a Markdown and HTML files#

from deepchecks.tabular.datasets.classification.adult import load_data, load_fitted_model
from deepchecks.tabular.suites import train_test_validation
model = load_fitted_model()
train, test = load_data()
ttvs = train_test_validation()

# run the suite and get a SuiteResult
result = ttvs.run(train_dataset=train, test_dataset=test, model=model)

# save the SuiteResult as a GitLab- or GitHub- compliant markdown
result.save_as_cml_markdown(file='report_gitlab.md', format='gitlab')
# a full html report - report_gitlab.html - is produced alongside report_gitlab.md
# its relative path to the .md file must stay consistent for cml to find it.

Use CML post the report to a Pull/Merge Request#

test-data-integrity:
  stage: test_data
  script:
    - dvc pull
    # say this command produces ./report_gitlab.md and ./report_gitlab.html
    - dvc repro train_test_validation
    # make cml make a comment in the PR/MR with the produced summary report file
    - cml comment create report_gitlab.md --publish --publish-native