User Guide

Here you can find the key concepts, structure, recommended flow, and dive in to many of the deepchecks functionalities.

General

This section section gives an overview about how to work with deepchecks: when to use it, our concepts and hierarchy, different ways to save the check and suite results, how to customize checks, suites, metrics, etc.

General

• When Should You Validate & Built-In Suites
  • Built-In Suites - API Reference
  • Typical Validation Scenarios
  • New Data: Single Dataset Validation
  • After Splitting the Data: Train-Test Validation
  • After Training a Model: Analysis & Validation
  • General Overview: Full Suite
• Deepchecks Hierarchy
  • Check
  • Condition
  • Suite
• Viewing Deepchecks Results
  • Main Methods for Showing Results
  • Alternative Methods for Showing Results (Troubleshooting)
• Exporting Deepchecks’ Results
  • Save Result as an HTML Report: save_as_html()
  • Export Results to JSON: to_json()
• Customizations
• Metrics Guide
  • Default Metrics
  • Alternative Metrics
  • List of Supported Strings
  • Custom Metrics
• Drift User Guide
  • What Is Distribution Drift?
  • Which Types of Drift Are There?
  • How Do You Detect Drift?
  • How Can I Use Deepchecks to Detect Drift?
  • What Can You Do in Case of Drift?
  • Code Examples
• Using Deepchecks In CI/CD
  • CI/CD In Machine Learning
  • Airflow Integration
  • GitHub Actions Integration

Tabular

Here you can see quickstarts of how to start working with deepchecks on tabular data, and much additional information related to the tabular supported use cases and customizations.

Tabular

• Tabular Quickstarts
  • Quickstarts:
• Tabular Tutorials
• The Dataset Object
  • Class Properties
  • Creating a Dataset
  • Useful Functions
  • Inferring Categorical Features
• Working with Models and Predictions
  • Supported Tasks and Predictions Format
  • Passing a Model
  • Using Pre-computed Predictions
• Feature Importance
  • What is Feature Importance?
  • Why Does Deepchecks Use Feature Importance?
  • How Does Deepchecks Get Feature Importance?
  • What if the Feature Importance Calculation Takes Too Long?
• Custom Check Templates
  • Single Dataset Check
  • Train Test Check
  • Model Only Check

Vision

Here you can see of how to start working with deepchecks on vision data, and much additional information related to the vision supported use cases and customizations.

Vision

• Computer Vision Tutorials
• The Vision Data Class
  • Common Class Parameters
  • Creating a VisionData Object
  • Adding Model Predictions
• Supported Tasks and Formats
  • Common Formats
  • Classification
  • Object Detection
  • Semantic Segmentation
  • Other Tasks
• Vision Properties
  • What Are Properties Used For?
  • Deepchecks’ Built-in Properties
  • Property Structure
  • Customizing the Checks’ Properties
• Custom Check Templates
  • Single Dataset Check
  • Train Test Check
  • Model Only Check

NLP

NLP

• NLP Quickstarts
  • Quickstarts:
• Supported Tasks and Formats
  • Supported Task Types
  • Supported Labels and Predictions Format

Integrations

Here you can see code examples for how to use deepchecks with various existing tools. Of course, deepchecks can easily be integrated with many additional tools, here you can find examples and code snippets for inspiration. Contributions to this docs section are very welcome!

Integrations

• Spark & Databricks
  • Loading the dataset
  • Defining deepchecks Dataset
  • Running the Integrity Suite
  • Building a Logistic Regression Model
  • Evaluating the Model Using Deepchecks Suites
• Pytest
  • Defining Pytest Fixtures
  • Implementing the Test
• H2O
• HuggingFace Transformers
  • Building a VisionData Object for the DETR Model
  • Load Pre-Made YOLOv5s
  • Benchmarking YOLOv5s Against DETR ResNet
• Airflow
  • Defining the Data & Model Loading Tasks
  • Defining the Integrity Report Task
  • Defining the Model Evaluation Task
  • Creating the DAG
• CML
  • Export SuiteResult as a Markdown and HTML files
  • Use CML post the report to a Pull/Merge Request
• JUnit
  • General Structure
  • Executing a Test Suite
• Weights & Biases (wandb)
  • Load Data
  • Run a Check
  • Observe CheckResult Display and Value
  • See Check to_wandb options
  • Run Suite and export to wandb

previous

Installation

next

When Should You Validate & Built-In Suites