AI Cleaning Workflow And Success Criteria

A stakeholder asks for monthly active users and retention by plan tier from a SaaS product-usage export covering 2025-01-01 to 2025-06-30. The CSV has 120k rows at event level, with mixed timestamp formats, null user_id values, and repeated events from replays. Cleaning is not a preliminary chore. Cleaning is the set of analytical decisions that determine what counts as activity, who counts as a user, and which records are admissible.

Your deliverable is audit-ready. You own a rerunnable cleaned dataset plus a change log that justifies every transformation. AI can draft code, enumerate edge cases, and propose validation checks. You ratify definitions, you choose thresholds, and you carry responsibility for downstream error when a cleaning choice distorts a metric.

Before touching code, you must define what clean enough means for this question. That definition becomes acceptance tests that the cleaned output must pass.

Quality decisions that change the answer

Every cleaning decision is a statistical intervention. It changes the data generating process you are pretending produced the table, and it moves estimates.

Missingness mechanism is the reason values are absent. If plan_tier is null because the billing table failed to join for a specific week, imputing the most common tier will fabricate segment counts and bias retention by tier. You must label missing as not measured, not recorded, or not applicable, and only then choose deletion, imputation, or a separate category. AI can suggest options, but it cannot infer mechanism from the column alone.

Deduplication rule is the business definition of duplicate. If replayed events share (user_id, event_name, event_ts) within 5 seconds, collapsing them may be correct for usage counts but wrong for latency analysis. When AI drops duplicates with drop_duplicates(), it silently chooses keys and precedence. You must specify keys, tie-breakers, and the metric impact.

Type coercion changes ordering and grouping. Parsing event_ts incorrectly can shift events across months and create a false MAU change. You must verify parse success rate and time zone assumption.