A leakage checklist built around availability time

Sort order alone cannot prevent leakage. For every prediction, determine which particular values were available before the decision. Then inspect fitting and selection for information taken from evaluation outcomes.

Use an availability ledger rather than relying on column names such as “historical” or “lagged.” Those labels do not prove the transformation is valid.

A worked boundary

At decision 600, x_600 is already known. A one-step label attached to row 599 resolves immediately before decision 600 and is eligible under that stated convention. A three-step label attached to row 598 resolves at 601 and is not eligible.

For labels attached to t and available at t+3, the last eligible training row at decision 600 is 597. The correct rule is the availability comparison, not a memorized gap length that you reuse after changing the target.

A completed availability ledger

Here is an original synthetic example for decision 600. Availability at 600 means the value is received immediately before that decision. A source that arrives afterward needs a different timestamp even if both events share the same coarse time bucket.

The last row concerns the revised version specifically. An earlier released version might be eligible if you retained it and can establish when it was available. Replacing that historical version with today's revised value changes the information used in the experiment.

For your own ledger, copy these columns and add the source, version and reason for each decision. Check the raw inputs of derived features too: a centered average inherits the availability of its latest required input.

Audit five places

Features: inspect source inputs, publication times, centered windows and revisions. A value describing an earlier period may have been published later.

Labels: exclude unresolved outcomes at every fitting boundary. Overlapping targets can also create dependence even when no information leaks.

Preprocessing: estimate transformation parameters on eligible training data. Applying a learned transformation later is different from fitting it using later rows.

Selection: record which features, windows and metrics were chosen after examining evaluation results. Early-only coefficient fitting does not undo later-informed selection.

Dataset construction: inspect whether the available universe or historical values were reconstructed using information unavailable then. State limitations when a point-in-time dataset cannot be established.

An impossible diagnostic

Use the outcome itself as a feature and a simple model can predict perfectly, even with chronological training and evaluation. This demonstrates why a clean split is insufficient. The input is unavailable at decision time.

Remove that feature from valid comparisons. A pipeline cannot make an inherently future-valued input legitimate.

Record the result

For each field, keep event time, availability time, decision time, eligibility and reason. Attach one concrete example at a boundary. This is easier to review than a general assurance that the project “avoids look-ahead.”

scikit-learn's common pitfalls and cross-validation guide provide implementation foundations. You remain responsible for the target and timing contract. The research lab supplies an original synthetic example and editable audit worksheet.