Data Quality Analysis: Turn Checks Into Same Day Fixes

Analyst reviewing missing and inconsistent data

Data quality analysis determines whether a dataset is fit for its intended use by measuring core dimensions like accuracy, completeness, and consistency, then prioritizing fixes that matter most to downstream analysis. The goal is a prioritized remediation list, not a perfect dataset. Before anything else, run a quick profile of your table: row counts, null rates, and distinct values will tell you where to look first.

TL;DR:

Profile new tables for null rates, distinct keys, ranges, and timestamps, then define business rules; sampling against trusted references is necessary to verify accuracy.

Choose the two or three dimensions tied to the dataset’s decision, and rank failures by affected records and business cost rather than rule count.

Avoid a blind average across quality dimensions; weight each score by its effect on the use case or cap the total at a critical dimension.

Start monitoring with 7–14 days of history for basic anomalies, extend baselines to months for seasonal data, and keep hard rules for business critical fields.

For legally or financially sensitive fields with uncertain accuracy, quarantine affected rows instead of imputing values, and verify repairs with before and after scores.

What data quality is and why it matters for analysis, ML, and reporting
Core DQ dimensions: operational definitions and detection patterns
How to assess: profiling, rule-based validation, sampling, and triage
Metrics and scoring: turning checks into per-dimension and dataset-level scores
Tools and approaches: rule-based checks, observability platforms, and ML methods
Remediation and cleaning: pragmatic fixes, documentation, and verification
Monitoring and observability: preventing drift and alert design
Standards and reference frameworks
A practitioner checklist for assessing an experiment results table
What the data quality conversation gets wrong
Reducing data quality problems at the source with Gostellar
FAQ
Sources

What data quality is and why it matters for analysis, ML, and reporting

"Fit for use" is the phrase that separates data quality from data perfection. A dataset can have messy formatting and still be fit for use in a weekly trend report, while the same dataset might be unusable for a machine learning model that needs clean, consistent features. Quality is contextual: it depends on what decision the data will drive, who will see the output, and how much error that decision can tolerate.

The attributes that define quality keep recurring across contexts: accuracy, completeness, consistency, timeliness, validity, and uniqueness. ISO/IEC 25012 formalizes these into a general data quality model, distinguishing characteristics that are inherent to the data itself from those that depend on the system storing or delivering it. That distinction matters in practice: a phone number field can be inherently valid (correct format) while being system-dependent unreliable (truncated by a legacy database column).

Poor data quality shows up as concrete failures, not abstract risk. A marketing dashboard reports inflated conversion rates because duplicate events double-count purchases. A churn model trained on incomplete customer records misses the signal that actually predicts churn. A/B test results get misread because timestamp drift between two systems makes "day 7" mean different things in each table. Industry research on the financial impact of poor data quality has flagged it as a material source of loss that grows more severe as organizations feed that data into AI systems.

Certain moments call for a dedicated data quality analysis rather than ad hoc spot checks:

Onboarding a new data source or vendor feed into your warehouse.
Before training or retraining a machine learning model on a dataset.
After any pipeline change: a new ETL job, a schema migration, or a third-party API update.
Before a report or dashboard goes in front of executives or external stakeholders.
When downstream users start questioning numbers that used to be trusted.

Each of these triggers deserves the same disciplined approach: profile first, define rules second, score third, then remediate. The sections below walk through exactly how.

Core DQ dimensions: operational definitions and detection patterns

Abstract dimensions only become useful once you can test for them. Here is what each one means operationally, and the checks that reveal when it fails.

Accuracy asks whether a value correctly represents the real-world fact it describes. You usually cannot verify accuracy from the dataset alone: it requires a reference source, like a verified customer record or a third-party address validator, against which you sample and compare.

Completeness asks whether required values are present. The standard check is a null rate per column, plus a check for silent defaults (zeros or empty strings masquerading as missing data).

Consistency asks whether the same fact is represented the same way across tables, systems, or time. Detection usually means reconciling totals between a source system and a downstream copy, or checking that a customer's country code matches across the CRM and the billing system.

Timeliness asks whether data is available and current enough for its intended use. A common pattern is to check the lag between the current time and the most recent timestamp in a table, then compare it against the refresh schedule the pipeline promises.

Validity asks whether values conform to a defined format, range, or business rule. Regex checks on email fields, range checks on dates, and enum checks on categorical fields all fall here.

Uniqueness asks whether each real-world entity appears once, not duplicated. The standard check compares a count of distinct keys against a total row count on the field that should be a primary key.

A few detection patterns cover most of these checks without specialized tooling: null-rate queries per column, distinct-count comparisons on key fields, cross-table reconciliation sums, and a simple "max timestamp minus now" lag calculation. These four queries, run on any new table, surface the majority of quality problems worth prioritizing.

Four common data profiling checks

Pro Tip: Run your null-rate and distinct-count checks together in one profiling pass. Completeness and uniqueness failures often share a root cause, like a broken upstream join.

A systematic mapping of data quality tool functions to formal dimensions finds that the relationship is many-to-many rather than one-to-one: a single regex validation function might support both validity and consistency checks, while a single dimension like accuracy often requires several different tool functions working together. This matters because it means you cannot buy a single feature and call a dimension "covered." You have to map your specific checks to the dimension they actually serve.

Failures also overlap in practice. A spike in null values might look like a completeness problem, but if it appeared the same day a new upstream service was deployed, the root cause is a validity or consistency break further up the pipeline. Treat a dimension failure as a symptom and trace it back before you remediate.

Not every dimension deserves equal attention on every dataset. A transactional finance table needs strict accuracy and uniqueness checks because errors there have direct monetary consequences. A marketing engagement log can tolerate looser accuracy but needs tight timeliness, since stale engagement data makes same-day personalization decisions worthless. Choose the two or three dimensions that matter most for the decision the dataset supports, and invest your checking effort there first.

How to assess: profiling, rule-based validation, sampling, and triage

A data quality assessment is a sequence of concrete steps, each producing an artifact you can hand to a stakeholder or feed into the next step.

Profile the schema and distributions. Pull column types, null rates, min and max values, cardinality per column, and a sample of distinct values. This single pass tells you where the obvious problems live before you write a single business rule.
Define business rules and constraints. Translate domain knowledge into testable conditions: order totals must be positive, a user ID must exist in the users table, a status field must be one of a fixed set of values.
Run conformance checks against those rules. Execute the rules across the full dataset or a representative slice, and record the pass and fail counts per rule.
Sample for accuracy and truth against reference data. Pull a random sample of rows and manually or programmatically verify them against a trusted source, like a verified customer database or an external validation API.
Triage findings by impact and volume. Rank failures by how many records they affect and how much downstream damage they cause, not simply by how many rules failed.
Map each issue back to a probable root cause. Was it a schema change, a timezone mismatch, a broken join, or a manual data entry error? The fix depends entirely on the cause.

Sampling deserves special attention because it is the only step that can validate accuracy, the dimension that profiling and rule checks cannot see on their own. A rule can confirm a date is well-formed; only a reference check can confirm the date is correct.

Pro Tip: Triage by multiplying the number of affected records by the business cost per record.

Metrics and scoring: turning checks into per-dimension and dataset-level scores

Checks only become actionable once they are expressed as numbers that stakeholders can track over time. Each dimension has a natural metric: a completeness ratio (non-null rows divided by total rows), a uniqueness failure rate (duplicate keys divided by total keys), and a validity pass percentage (rows meeting format rules divided by total rows).

Dimension	Example metric	Typical expression
Completeness	Completeness ratio	Non-null values ÷ total values
Uniqueness	Duplicate rate	Duplicate keys ÷ total keys
Validity	Validity pass rate	Conforming rows ÷ total rows
Timeliness	Freshness lag	Current time minus latest timestamp
Consistency	Reconciliation variance	Source total minus downstream total

Rolling individual metrics into a single dataset-level score requires a judgment call, and a blind average across dimensions is usually the wrong one. A dataset can score 98% on completeness and 60% on accuracy, and a straight average would mask the accuracy problem that actually matters for the business decision riding on that data. Weighted scoring, where each dimension's weight reflects how much that dimension affects the specific use case, produces a far more honest number.

A few practical options for combining scores:

A simple weighted average, with weights set by a data owner based on business priority.
A worst-dimension floor, where the overall score cannot exceed the lowest-scoring critical dimension.
A tiered classification, similar to sorting records into high, medium, and low quality bands rather than forcing a single percentage.

Once scores exist, present them where people will actually see them. A trend line per dimension over time catches slow degradation that a single snapshot misses. A heatmap across tables and dimensions helps a data team spot which tables need attention without reading a report line by line. SLA indicators, simple red, yellow, green markers against an agreed threshold, let non-technical stakeholders know at a glance whether a dataset is safe to use today.

Tools and approaches: rule-based checks, observability platforms, and ML methods

Four broad approaches cover most data quality work, and they are not mutually exclusive.

Manual SQL rules are the cheapest to start with and the easiest to explain to a stakeholder, but they only catch the specific failure modes you thought to write a rule for.
Profiling tools automate the schema and distribution scan from the assessment workflow, surfacing anomalies you did not anticipate.
Observability platforms extend profiling into continuous monitoring, tracking metrics over time and alerting on drift rather than requiring a manual rerun.
ML and ensemble classifiers, including Random Forest and XGBoost-style models, learn patterns across many columns at once and catch problems that a fixed threshold rule misses entirely.

ML methods earn their place at scale, particularly for hidden missing values that do not look like nulls (a default value, a placeholder string, or an implausible but technically valid number) and for datasets too large to review with manual sampling. Recent research on modern data quality assessment describes a shift from static, threshold-based checks toward ML-based frameworks that classify records into quality tiers, which scale better across large, heterogeneous pipelines than any fixed rule set.

A systematic mapping of tool functions to formal DQ dimensions shows the relationship is many-to-many: a single tool function often supports multiple dimensions, and a single dimension typically requires multiple functions to assess completely.

The practical implication is that no single tool "covers" a dimension. A validity check on an email format also partially supports consistency; a duplicate-detection function supports both uniqueness and, indirectly, accuracy. Build your assessment around the dimensions that matter for your use case, then select whichever combination of rules, profiling, and ML detection actually serves each one, rather than buying a tool and assuming its feature list maps cleanly to your checklist.

Remediation and cleaning: pragmatic fixes, documentation, and verification

Fixing a data quality problem is only half the job. Doing it in a way that is reversible and auditable is what keeps the fix from becoming tomorrow's mystery.

Imputation fills missing values with a reasonable estimate (a mean, a median, or a model-based prediction), appropriate when the missing rate is low and the field is not safety- or legally critical.
Deduplication merges or removes duplicate records, appropriate once you have confirmed which record in a duplicate pair holds the more accurate or more recent value.
Backfills reprocess historical data through a corrected pipeline, appropriate when a bug affected a known date range and the source data needed to reprocess it still exists.
Normalization standardizes formats, such as collapsing "NY," "N.Y.," and "New York" into one canonical value, appropriate for any field feeding a join or a groupby.

Safe remediation practice depends on a few habits that cost little and save a lot of pain later. Keep an immutable log of the original data before any transformation runs, so a bad fix can always be reversed. Record every transformation in a metadata layer or data catalog, including who or what made the change and why. Version your cleaning logic the same way you version application code, so a regression in the cleaning rules is as easy to trace as a regression in a feature.

Pro Tip: When accuracy is uncertain and the field has legal or financial weight, quarantine the affected rows instead of guessing. A quarantined row that waits for manual review is safer than an imputed value that turns out wrong.

Verification closes the loop. Capture the dimension scores before a fix and the same scores after, and treat a meaningful improvement as the bar for calling the fix successful. For fixes applied inside a pipeline, add a regression test that would have caught the original failure, so the same bug cannot silently reappear after the next schema change.

Monitoring and observability: preventing drift and alert design

A data quality assessment is a snapshot. Monitoring is what keeps that snapshot from going stale the week after you run it.

Baselines need enough history to mean something. Basic anomaly detection on a metric like null rate or row count can work with 7 to 14 days of history, but a metric with weekly or yearly seasonality, like transaction volume around a holiday, needs a longer baseline measured in months to avoid flagging normal seasonal swings as anomalies.

Set a short baseline window for fast-moving operational metrics, and a multi-month window for anything with seasonal patterns.
Prioritize alerts by the business impact of the metric, not just by how far it deviated from baseline.
Use quality gates in your pipeline the same way you would use a CI test suite in software: a dataset that fails a critical check does not move downstream until someone reviews it.
Build separate dashboard views for data owners, who need a simple pass or fail signal, and engineers, who need the underlying metric trends to debug root causes.

Fixed thresholds tend to break down as data drifts over time. Research on automated assessment finds that adaptive, distribution-based thresholds reduce false alerts and catch genuine problems more reliably than static cutoffs that were calibrated once and never revisited. Combine that adaptive layer with a handful of hard-coded rule-based invariants for anything business-critical, like primary key uniqueness or a legally required field, since those checks should never flex no matter what the distribution says.

Standards and reference frameworks

Grounding your terminology in a published standard saves a data team from reinventing definitions every time a new analyst joins.

ISO/IEC 25012 defines a general data quality model and a taxonomy of characteristics, split between qualities inherent to the data and qualities dependent on the system storing it. Use it to agree on what "accuracy" or "completeness" means before your team starts arguing about it in a Slack thread.
ISO/IEC 25024 builds on that model with guidance for quantitative measurement, but it deliberately avoids fixing universal thresholds. The standard expects you to define your own target values per system and context, which matches how quality actually works in practice: a 2% null rate might be fine for one field and unacceptable for another.
ML-driven frameworks that classify records into quality tiers extend these ideas into automated, large-scale pipelines, profiling and scoring records without a human writing every rule by hand.

The practical sequence is to align your team's vocabulary to ISO's definitions first, then adapt the actual measurement thresholds to your own context, since no standard can tell you what "good enough" means for your specific dataset and decision.

A practitioner checklist for assessing an experiment results table

Running a data quality analysis on an A/B test results table follows the same sequence every time:

Scope the table: identify the critical fields (user_id, variant, conversion_event, timestamp) that the analysis depends on.
Profile each field for null rates, distinct counts, and value ranges before writing a single rule.
Define invariants: every user_id should map to exactly one variant, and timestamp should fall within the experiment's active window.
Run conformance checks against those invariants across the full table.
Sample-truth a batch of conversion events against raw event logs to confirm accuracy.
Prioritize failures by how many users or conversions they touch.
Fix or quarantine affected rows, documenting the decision.
Verify the fix with before and after completeness and validity scores.
Set up ongoing monitoring so the next experiment catches the same failure before it reaches analysis.

Common failure patterns on this type of table include users appearing in two variants at once (a randomization bug), conversion events logged after the experiment window closed, and timestamp mismatches between the assignment event and the conversion event caused by clock drift across services. For a closer look at interpreting the resulting numbers once the table is clean, see our guide to analyzing test results.

What the data quality conversation gets wrong

Most data quality advice treats every dimension as equally urgent, which sends teams chasing completeness percentages on fields nobody downstream actually uses. The better instinct is to work backward from the decision the data supports and check only what that decision depends on. A dashboard that drives a single weekly budget call needs far less rigor than a table feeding a pricing model.

The other overrated habit is treating a quality score as a finish line. A dataset that scores 95% today can drift to 80% within weeks if nothing monitors it, and a one-time cleanup without a monitoring layer behind it is a guarantee that the same work repeats itself in a quarter. Score, fix, and monitor are one continuous loop, not three separate projects.

If I had to pick one place to start, it would be timeliness and uniqueness checks on whatever table feeds your most-used report, because those two dimensions are the cheapest to check and the most common source of numbers that quietly stop matching reality. Everything else can wait until that foundation holds.

— Juan

Reducing data quality problems at the source with Gostellar

A large share of the data quality problems we have described in this guide start before a single row ever lands in a warehouse: a tracking script fires twice, a variant gets logged inconsistently, or a conversion event never fires at all. Fixing those problems downstream with cleaning rules works, but it is still cleanup. Preventing them at the instrumentation layer is faster.

Gostellar

Our A/B testing platform uses a lightweight script designed to reduce event duplication and delayed firing that can affect experiment data quality. The platform includes goal tracking and real-time analytics aimed at maintaining consistency in key fields from the moment an experiment launches, helping reduce reconciliation work later in the data quality analysis process. For teams using popular site builders, the platform supports consistent event schemas across tests as part of its data tracking approach.

If instrumentation consistency is the gap in your current experiment data, see our plans and pricing to find the tier that fits your traffic volume, including a free Sandbox plan for teams tracking under 25,000 monthly users.

FAQ

What is data quality analysis?

Data quality analysis is the process of measuring a dataset against dimensions like accuracy, completeness, consistency, timeliness, validity, and uniqueness to determine whether it is fit for a specific use. The output is typically a scored report that prioritizes which issues to fix first based on their impact on the decision the data supports.

What are the 5 elements of data quality?

Definitions vary slightly by source, but a common version centers on accuracy, completeness, consistency, timeliness, and validity, with uniqueness often added as a sixth. ISO/IEC 25012 defines a broader taxonomy of characteristics that these five commonly cited elements draw from.

What are the 4 categories of data quality?

A widely used grouping splits data quality characteristics into intrinsic (accuracy, objectivity), contextual (relevance, timeliness), representational (interpretability, consistency), and accessibility (availability, security) categories. These categories help teams organize checks by what they measure rather than treating every dimension as a single flat list.

What are the 7 C's of data quality?

There is no single standardized "7 C's" framework recognized across the industry the way ISO's dimensions are; different practitioners and vendors use varying lists under that name. Rather than relying on an informal mnemonic, it is more reliable to anchor your checks to a documented model like ISO/IEC 25012 or ISO/IEC 25024.

How often should I run a data quality analysis?

Run a full assessment whenever you onboard a new data source, before training a machine learning model, and after any pipeline or schema change, then layer continuous monitoring on top rather than relying on periodic manual checks alone. A baseline monitoring window of 7 to 14 days is typically enough to start catching basic anomalies, with longer windows needed for seasonal patterns.

Try Stellar A/B Testing for Free!