Edit Checks: What They Catch and What They Create

Part of Clinical Data Management and Integrity: Designing the Data Collection Instrument.

About 5 minutes. Free, in your language, no account needed.

Why this matters

Edit checks are the automated rules that flag values as impossible, implausible or inconsistent. They are the cheapest quality mechanism available and the easiest to over-apply.

A well specified set catches real errors at the point of entry, when the participant may still be in the building and the source is at hand. A poorly specified set generates thousands of queries that consume site goodwill and resolve into nothing, and the cost of that is not merely wasted effort. It trains sites to treat queries as noise, which degrades the response to the queries that matter.

The three kinds

Univariate checks test one field. A systolic blood pressure of 900. A date in the future. A weight of 4 kilograms in an adult study. These catch typing errors and are cheap and reliable.

Cross-field checks test relationships. An end date before a start date. A pregnancy record for a participant recorded as male. A dose recorded on a day the participant is recorded as having withdrawn. These catch real inconsistencies and are where most of the value is.

Cross-form and cross-visit checks test coherence across the record. An adverse event with an onset before enrollment. A concomitant medication for a condition never recorded. A laboratory value marked clinically significant with no corresponding adverse event. These catch the most meaningful problems and are the most expensive to specify.

Hard and soft

A hard check prevents saving. A soft check warns and permits the user to proceed, usually requiring a confirmation that is itself recorded.

The rule is narrow and important: hard-block only what is genuinely impossible. A date of birth after the enrollment date is impossible and may be blocked. A systolic pressure of 300 is implausible and may be real, and blocking it guarantees a false record for the participant who actually had it.

The common error is hard-blocking the implausible because it feels safer. It is not safer. It shifts the error from a flagged unusual value to an unflagged normal-looking one, and the second is undetectable.

The cost of a check

Every check has a cost paid by someone else, and the specification should be written with the resulting query in front of you.

Before adding a check, three questions: What will the site actually do when it fires? Can the site resolve it from information it has? And what would we do differently if it fired a hundred times?

A check whose typical resolution is "value confirmed as correct" is a check that should not exist. It is finding nothing and costing something each time.

The measure that matters

Not the number of checks, and not the number of queries. The proportion of queries that result in a data change.

A check firing 200 times and producing 4 changes has a two percent yield. It is not protecting data; it is spending 196 units of site attention to find four things, and the attention would have been better spent elsewhere.

Reviewing check yield after the first few months of enrollment, and retiring the checks that find nothing, is among the highest-value things a data manager does and is almost never done, because the checks were signed off during set-up and revisiting them feels like reopening a settled question.

In practice

A data manager reviews check performance three months into enrollment.

Of 180 active checks, 41 have never fired, which is fine and tells her nothing yet. 12 have fired more than 50 times each, accounting for 68 percent of all queries raised.

Of those 12, nine have a yield below five percent: they fire, the site confirms the value, the query closes with no change. One of them fires on any laboratory value outside the reference range, in a population where abnormal values are the expected clinical picture.

She retires seven checks, narrows two, and keeps three that have low yield but guard the primary endpoint, where a low yield is an acceptable price.

Query volume falls by just over half. The queries that remain are the ones sites actually read.

What this looks like when it goes wrong

A study hard-blocks any adverse event with an onset date before the first dose, on the reasoning that such an event is not treatment emergent and belongs in medical history.

The reasoning is sound for most events. It is wrong for one case that occurs regularly: an event beginning before first dose and worsening after it, which is a reportable adverse event with a pre-dose onset.

Sites cannot enter the true onset date. They enter the first dose date instead, because that is what the system accepts.

The database now shows a cluster of adverse events with onset exactly on the day of first dose. It looks like a signal. It is an artifact of the block, and it took a safety review and several weeks to establish that, during which a genuine safety question was under investigation that did not exist.

A soft warning would have produced the true dates with a flag.

Key takeaways