The Data Lifecycle, From Source to Lock

Part of Clinical Data Management and Integrity: What Data Management Owns.

About 5 minutes. Free, in your language, no account needed.

Why this matters

Data does not appear in a database. It travels, and it changes form at every step of the journey. Each transformation is a place where meaning can be lost, and the failures that matter almost always occur at a boundary rather than inside a step.

Knowing the boundaries tells you where to look.

The journey

Something happens to a participant. A symptom, a measurement, a dose, a visit.

Somebody records it. In a clinical note, on a printed worksheet, on an instrument's own output, or directly into an electronic system. This record is the source, and what counts as source is a decision that must be made deliberately rather than discovered later.

The value is transcribed into the case report form, unless the form is itself the source. Every transcription is an opportunity for a value to change, which is why direct entry is preferred where it is genuinely feasible.

The system applies edit checks, flagging values that are impossible, implausible, or inconsistent with other values.

Discrepancies become queries, which go to the site, which answers them, and the answer either changes a value or explains why the value is right.

External data arrives separately from laboratories, imaging vendors, devices and electronic outcome systems, on its own schedule and in its own format, and must be reconciled against the clinical database.

Adverse events are coded to a standard dictionary, and medications to another.

The data is mapped to a standard structure for analysis and submission.

The database is locked, the treatment allocation is revealed where the study is blinded, and the analysis runs.

Where the failures actually live

At the boundaries, consistently:

Between what happened and what was recorded. A value never written down cannot be recovered, and nothing downstream will reveal its absence.

Between source and case report form. Transcription errors, and worse, transcription that interprets. A source note reading "BP 140/90, pt anxious" becomes a blood pressure value, and the clinical context that would have explained it is gone.

Between external data and the clinical database. The most common serious reconciliation gap is between the serious adverse events recorded in the safety database and those in the clinical database, because they are entered by different people through different routes at different times.

Between the collected structure and the analysis structure. A mapping decision made by someone who does not know what the analysis requires can silently discard the distinction the analysis depends on.

Between lock and unblinding. Anything discovered after unblinding is discovered by people who now know the allocation, and a correction made at that point, however honest, cannot be shown to be independent of that knowledge.

The direction of travel is one way

This is the property that governs everything.

Each step forward is cheap and each step back is expensive, and the cost rises steeply. Correcting an instrument design during protocol development costs a conversation. Correcting it after the first participant is enrolled costs an amendment. Correcting it after database lock costs an unlock, and an unlock is an event a regulator will ask about.

Everything in this course is, in one way or another, an argument for spending effort early.

In practice

A data manager joins a study at first participant enrolled and is asked to focus on query resolution.

Her first week is spent instead on a data flow diagram: every source of data, its route into the database, its format, its frequency, its owner, and the reconciliation that confirms it arrived complete.

The diagram has nineteen sources. Four have no defined reconciliation, meaning nobody would notice if a transfer silently dropped records. Two arrive in a format whose mapping has not been specified, so the specification will be written by whoever happens to do it under time pressure later. One vendor's data includes a field the analysis requires and the transfer specification does not mention.

None of this is visible in a query listing. All of it is more consequential than any query, and all of it is cheap to fix now and expensive to fix at lock.

What this looks like when it goes wrong

A study collects tumor assessments from an imaging vendor, which transfers results monthly.

In month seven a configuration change at the vendor causes assessments for one site to be transmitted with a site identifier that the clinical database does not recognize. The records arrive, fail to match a participant, and are written to an exception file that nobody reads.

Eleven months later, at the pre-lock reconciliation, 48 assessments are missing for that site. The records exist at the vendor and can be recovered, so no data is lost.

What is lost is eleven months of not knowing, during which the study's monitoring of its own primary endpoint was working from incomplete data, and interim decisions were made on that basis.

The reconciliation that would have caught it in month seven was defined but scheduled quarterly and performed only when someone had time.

Key takeaways