Skip to main content

Documentation Index

Fetch the complete documentation index at: https://openmetadata-feat-feat-2mbfixtestexui.mintlify.app/llms.txt

Use this file to discover all available pages before exploring further.

​
Column-Level Test Definitions

Column-level tests validate properties of specific columns. For table-level tests and importing instructions, see the Test Definitions Reference.

​
Column-Level Tests

​
ColumnValuesToBeNotNull

Validates that a column contains no null or missing values. Parameters:
  • column (str, required): Name of the column to validate
Example: 
from metadata.sdk.data_quality import ColumnValuesToBeNotNull

# Email column must not contain nulls
test = ColumnValuesToBeNotNull(column="email")
Use Cases:
  • Ensure required fields are populated
  • Validate data completeness
  • Enforce NOT NULL constraints

​
ColumnValuesToBeUnique

Validates that all values in a column are unique with no duplicates. Parameters:
  • column (str, required): Name of the column to validate
Example: 
from metadata.sdk.data_quality import ColumnValuesToBeUnique

# Customer ID must be unique
test = ColumnValuesToBeUnique(column="customer_id")
Use Cases:
  • Validate primary keys
  • Ensure unique identifiers
  • Detect duplicate records

​
ColumnValuesToBeInSet

Validates that all values in a column belong to a specified set. Parameters:
  • column (str, required): Name of the column to validate
  • allowed_values (list[str], required): List of acceptable values
Example: 
from metadata.sdk.data_quality import ColumnValuesToBeInSet

# Status must be one of these values
test = ColumnValuesToBeInSet(
    column="status",
    allowed_values=["active", "inactive", "pending"]
)

# Country codes must be valid
test = ColumnValuesToBeInSet(
    column="country_code",
    allowed_values=["US", "UK", "CA", "AU", "DE"]
)
Use Cases:
  • Validate enum values
  • Enforce categorical constraints
  • Validate lookup values

​
ColumnValuesToBeNotInSet

Validates that column values do not contain any forbidden values. Parameters:
  • column (str, required): Name of the column to validate
  • forbidden_values (list[str], required): List of values that must not appear
Example: 
from metadata.sdk.data_quality import ColumnValuesToBeNotInSet

# Prevent test email addresses
test = ColumnValuesToBeNotInSet(
    column="email",
    forbidden_values=["test@test.com", "admin@admin.com", "noreply@example.com"]
)
Use Cases:
  • Detect test data in production
  • Blacklist invalid values
  • Filter out placeholder values

​
ColumnValuesToMatchRegex

Validates that column values match a specified regular expression pattern. Parameters:
  • column (str, required): Name of the column to validate
  • regex (str, required): Regular expression pattern
Example: 
from metadata.sdk.data_quality import ColumnValuesToMatchRegex

# Validate email format
test = ColumnValuesToMatchRegex(
    column="email",
    regex=r"^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}$"
)

# Validate phone number format (US)
test = ColumnValuesToMatchRegex(
    column="phone",
    regex=r"^\+?1?\d{10}$"
)

# Validate date format (YYYY-MM-DD)
test = ColumnValuesToMatchRegex(
    column="date_string",
    regex=r"^\d{4}-\d{2}-\d{2}$"
)
Use Cases:
  • Validate data format consistency
  • Ensure pattern compliance
  • Detect malformed data

​
ColumnValuesToNotMatchRegex

Validates that column values do not match a forbidden regular expression pattern. Parameters:
  • column (str, required): Name of the column to validate
  • regex (str, required): Regular expression pattern that values must NOT match
Example: 
from metadata.sdk.data_quality import ColumnValuesToNotMatchRegex

# Prevent test emails
test = ColumnValuesToNotMatchRegex(
    column="email",
    regex=r".*@test\.com$"
)

# Detect test data markers
test = ColumnValuesToNotMatchRegex(
    column="name",
    regex=r"^test.*"
)
Use Cases:
  • Detect test data patterns
  • Prevent specific formats
  • Identify security risks

​
ColumnValuesToBeBetween

Validates that all values in a column fall within a specified numeric range. Parameters:
  • column (str, required): Name of the column to validate
  • min_value (float, optional): Minimum acceptable value
  • max_value (float, optional): Maximum acceptable value
Example: 
from metadata.sdk.data_quality import ColumnValuesToBeBetween

# Age should be reasonable
test = ColumnValuesToBeBetween(
    column="age",
    min_value=0,
    max_value=120
)

# Price must be positive
test = ColumnValuesToBeBetween(
    column="price",
    min_value=0.01,
    max_value=999999.99
)

# Percentage values
test = ColumnValuesToBeBetween(
    column="completion_rate",
    min_value=0,
    max_value=100
)
Use Cases:
  • Validate numeric constraints
  • Detect outliers
  • Ensure value ranges

​
ColumnValueMaxToBeBetween

Validates that the maximum value in a column falls within a specified range. Parameters:
  • column (str, required): Name of the column to validate
  • min_value (float, optional): Minimum acceptable maximum value
  • max_value (float, optional): Maximum acceptable maximum value
Example: 
from metadata.sdk.data_quality import ColumnValueMaxToBeBetween

# Max temperature should be reasonable
test = ColumnValueMaxToBeBetween(
    column="temperature",
    min_value=-50,
    max_value=50
)

# Highest score should not exceed 100
test = ColumnValueMaxToBeBetween(
    column="score",
    min_value=90,
    max_value=100
)
Use Cases:
  • Monitor data ranges
  • Detect upper outliers
  • Validate maximum constraints

​
ColumnValueMinToBeBetween

Validates that the minimum value in a column falls within a specified range. Parameters:
  • column (str, required): Name of the column to validate
  • min_value (float, optional): Minimum acceptable minimum value
  • max_value (float, optional): Maximum acceptable minimum value
Example: 
from metadata.sdk.data_quality import ColumnValueMinToBeBetween

# Min temperature validation
test = ColumnValueMinToBeBetween(
    column="temperature",
    min_value=-50,
    max_value=0
)

# Minimum age should be at least 18
test = ColumnValueMinToBeBetween(
    column="age",
    min_value=18,
    max_value=25
)
Use Cases:
  • Monitor lower bounds
  • Detect lower outliers
  • Validate minimum constraints

​
ColumnValueMeanToBeBetween

Validates that the mean (average) value falls within a specified range. Parameters:
  • column (str, required): Name of the column to validate
  • min_value (float, optional): Minimum acceptable mean value
  • max_value (float, optional): Maximum acceptable mean value
Example: 
from metadata.sdk.data_quality import ColumnValueMeanToBeBetween

# Average rating should be between 3 and 4.5
test = ColumnValueMeanToBeBetween(
    column="rating",
    min_value=3.0,
    max_value=4.5
)

# Average response time monitoring
test = ColumnValueMeanToBeBetween(
    column="response_time_ms",
    min_value=100,
    max_value=500
)
Use Cases:
  • Statistical validation
  • Detect data drift
  • Monitor averages

​
ColumnValueMedianToBeBetween

Validates that the median value falls within a specified range. Parameters:
  • column (str, required): Name of the column to validate
  • min_value (float, optional): Minimum acceptable median value
  • max_value (float, optional): Maximum acceptable median value
Example: 
from metadata.sdk.data_quality import ColumnValueMedianToBeBetween

# Median salary range
test = ColumnValueMedianToBeBetween(
    column="salary",
    min_value=50000,
    max_value=75000
)
Use Cases:
  • Robust central tendency checks
  • Detect skewed distributions
  • Monitor typical values

​
ColumnValueStdDevToBeBetween

Validates that the standard deviation falls within a specified range. Parameters:
  • column (str, required): Name of the column to validate
  • min_value (float, optional): Minimum acceptable standard deviation
  • max_value (float, optional): Maximum acceptable standard deviation
Example: 
from metadata.sdk.data_quality import ColumnValueStdDevToBeBetween

# Response time variability
test = ColumnValueStdDevToBeBetween(
    column="response_time",
    min_value=0,
    max_value=100
)
Use Cases:
  • Detect unexpected variability
  • Monitor data consistency
  • Validate distribution stability

​
ColumnValuesSumToBeBetween

Validates that the sum of all values falls within a specified range. Parameters:
  • column (str, required): Name of the column to validate
  • min_value (float, optional): Minimum acceptable sum
  • max_value (float, optional): Maximum acceptable sum
Example: 
from metadata.sdk.data_quality import ColumnValuesSumToBeBetween

# Total revenue validation
test = ColumnValuesSumToBeBetween(
    column="revenue",
    min_value=1000000,
    max_value=5000000
)
Use Cases:
  • Validate totals
  • Monitor aggregates
  • Detect unexpected volumes

​
ColumnValuesMissingCount

Validates the count of missing or null values. Parameters:
  • column (str, required): Name of the column to validate
  • missing_count_value (int, optional): Expected number of missing values
  • missing_value_match (list[str], optional): Additional strings to treat as missing
Example: 
from metadata.sdk.data_quality import ColumnValuesMissingCount

# Expect exactly 100 missing values
test = ColumnValuesMissingCount(
    column="optional_field",
    missing_count_value=100
)

# Treat custom strings as missing
test = ColumnValuesMissingCount(
    column="status",
    missing_value_match=["N/A", "Unknown", ""]
)
Use Cases:
  • Monitor data completeness
  • Track missing data patterns
  • Validate optional fields

​
ColumnValueLengthsToBeBetween

Validates that string lengths fall within a specified range. Parameters:
  • column (str, required): Name of the column to validate
  • min_length (int, optional): Minimum acceptable string length
  • max_length (int, optional): Maximum acceptable string length
Example: 
from metadata.sdk.data_quality import ColumnValueLengthsToBeBetween

# Username length validation
test = ColumnValueLengthsToBeBetween(
    column="username",
    min_length=3,
    max_length=20
)

# Description length constraints
test = ColumnValueLengthsToBeBetween(
    column="description",
    min_length=10,
    max_length=500
)
Use Cases:
  • Validate string constraints
  • Prevent truncation
  • Ensure format compliance

​
ColumnValuesToBeAtExpectedLocation

Validates that a specific value appears at an expected row position. Parameters:
  • column (str, required): Name of the column to validate
  • expected_value (str, required): The exact value expected
  • row_index (int, optional): Zero-based row position (default: 0)
Example: 
from metadata.sdk.data_quality import ColumnValuesToBeAtExpectedLocation

# First row should have ID "1"
test = ColumnValuesToBeAtExpectedLocation(
    column="id",
    expected_value="1",
    row_index=0
)

# Top ranked item
test = ColumnValuesToBeAtExpectedLocation(
    column="rank",
    expected_value="first",
    row_index=0
)
Use Cases:
  • Validate sorted data
  • Check ordered results
  • Verify specific positions

​
Customizing Tests

All tests support customization through fluent methods: 
from metadata.sdk.data_quality import ColumnValuesToBeNotNull

test = ColumnValuesToBeNotNull(column="email") \
    .with_name("email_completeness_check") \
    .with_display_name("Email Completeness Validation") \
    .with_description("Ensures all customer records have email addresses") \
    .with_compute_row_count(True)
Or pass values directly to the constructor: 
test = ColumnValuesToBeNotNull(
    column="email",
    name="email_completeness_check",
    display_name="Email Completeness Validation",
    description="Ensures all customer records have email addresses"
)

​
Next Steps