Stop Writing .isnull() - Audit Your Dataset in One Line with OMR

Every time I start a new data project, I write the same boilerplate:

df.isnull().sum()
df.duplicated().sum()
df.dtypes
df.describe()
df[df['age'] < 0]
# ... 40 more lines

After doing this for the hundredth time, I built OMR (Omni Data Refinement) - a Python library that replaces all of that with a single call.

What is OMR?

OMR is an open-source Python framework for dataset quality, validation, profiling, and monitoring. Think of it as a health check for your data - before you do any ML, EDA, or transformation, you should know how healthy your data actually is.

Installation

pip install omni-data-refinement

Only needs pandas, numpy, and rich. No cloud, no LLMs.

The Core Feature: Health Score

import pandas as pd
from omr import Dataset

df = pd.read_csv("your_data.csv")
report = Dataset(df).health()
print(report.score)  # e.g. 87/100

You get a 0-100 quality score covering 5 dimensions:

Auto-Cleaning

dataset = Dataset(df)
dataset.clean()
dataset.explain_changes()  # See exactly what was fixed

OMR auto-resolves missing values, duplicates, and type mismatches - and gives you a full transformation log so nothing is a black box.

Schema Validation

from omr import schemas

schema = {
    "age": schemas.PositiveInteger(max=120),
    "salary": schemas.PositiveFloat(min=10000),
    "status": schemas.OneOf("active", "inactive"),
    "email": schemas.Email()
}

dataset.validate(schema)

Drift Detection

Compare your current dataset against production or a previous version:

prod_dataset = Dataset(pd.read_csv("prod_data.csv"))
dataset.compare(prod_dataset)

Uses PSI, KS Test, and JS Divergence under the hood.

Statistical Analysis

dataset.analyze()
# Detects: outliers, multicollinearity, skewness, class imbalance, zero variance

Column Profiling (replaces .describe())

dataset.profile()

Unlike .describe() which only covers numeric columns, OMR profiles every column - numeric, categorical, and boolean - in a clean terminal table.

The Fluent API

Chain everything together:

clean_df = (Dataset(df)
    .health()
    .clean()
    .analyze()
    .export())
# Exports HTML, Markdown, or JSON report

Why I Built This

The goal is for OMR to become the first thing you run after pd.read_csv() - not a replacement for Pandas, but the quality layer on top of it.

Links

• PyPI: pip install omni-data-refinement
• GitHub: https://github.com/Omar-Alshafai2/omni-data-refinement
• Docs: https://Omar-Alshafai2.github.io/omni-data-refinement/
• Examples: https://Omar-Alshafai2.github.io/omni-data-refinement/examples/

What data quality problems do you run into most? I would love to know what to build next.