Dataset splitting

This executable notebook compares QRT’s fixed temporal, expanding, rolling, and purged splitters on bundled SPY history. q.dataset.TimeSeriesSplit delegates positional fold construction to scikit-learn; QRT adds aligned named partitions, roles, metadata, diagnostics, and leakage controls.

Import dataset and sklearn APIs

The bundled dataset keeps the example offline and reproducible. We use sklearn’s StandardScaler inside QRT’s split-aware transform pipeline.

from tempfile import TemporaryDirectory

import pandas as pd
from sklearn.preprocessing import StandardScaler

import qrt as q

Build a realistic aligned dataset

We use roughly two years from the end of the bundled SPY daily history. Features are backward-looking and built with q.indicator. q.label.fixed_horizon creates the next five-session return classes and endpoints, and q.label.sample_weights attributes realized returns across overlapping labels.

The dataset’s metadata holds row-aligned context that must remain available without becoming a model feature. Here, symbol is descriptive context, while label_end_time is operational: PurgedTimeSeriesSplit reads it to remove training labels that overlap each test period, and audit_splits reads it to verify that leakage is absent. TemporalSplit and ordinary TimeSeriesSplit preserve this metadata in their partition views but do not use it to choose rows.

spy = q.data.datasets.load("spy").dropna(subset=["close", "volume"]).tail(520).copy()
returns = q.indicator.returns(spy["close"], periods=1)
features = pd.DataFrame(
    {
        "return_1d": returns,
        "momentum_20d": q.indicator.momentum(spy["close"], window=20),
        "volatility_20d": q.indicator.rolling_volatility(returns, window=20),
        "volume_ratio_20d": q.indicator.volume_ratio(spy["volume"], window=20),
    },
    index=spy.index,
).dropna()

labels = q.label.fixed_horizon(
    prices= spy["close"],
    horizon= 5,
    events=features.index,
)

index = labels.index

metadata = (
    q.label.purging_metadata(spy.index, labels["end_time"])
    .rename(columns={"end_time": "label_end_time"})
    .assign(symbol="SPY")
)

weights = q.label.sample_weights(spy["close"], labels["end_time"])

dataset = q.dataset.Dataset(
    X=features.loc[index],
    y=labels,
    sample_weight=weights,
    metadata=metadata,
)

print(f"{len(dataset):,} sessions from {dataset.index.min():%Y-%m-%d} to {dataset.index.max():%Y-%m-%d}")

dataset.to_pandas().head()
495 sessions from 2024-07-25 to 2026-07-16
return_1d momentum_20d volatility_20d volume_ratio_20d end_time return threshold label sample_weight label_end_time start_position end_position max_end_time max_end_position overlaps_previous next_safe_time next_safe_position symbol
event_time
2024-07-25 -0.005210 -0.013015 0.008333 1.490345 2024-08-01 0.008544 0.0 1 0.040633 2024-08-01 20 25 2024-08-01 25 False 2024-08-02 26 SPY
2024-07-26 0.011200 -0.003533 0.008734 1.267490 2024-08-02 -0.021196 0.0 -1 0.757228 2024-08-02 21 26 2024-08-02 26 True 2024-08-05 27 SPY
2024-07-29 0.000588 0.000993 0.008689 0.962946 2024-08-05 -0.050261 0.0 -1 2.838338 2024-08-05 22 27 2024-08-05 27 True 2024-08-06 28 SPY
2024-07-30 -0.005067 -0.006125 0.008750 1.104581 2024-08-06 -0.036624 0.0 -1 2.381440 2024-08-06 23 28 2024-08-06 28 True 2024-08-07 29 SPY
2024-07-31 0.016255 0.003278 0.009387 1.455946 2024-08-07 -0.058369 0.0 -1 2.329309 2024-08-07 24 29 2024-08-07 29 True 2024-08-08 30 SPY

Fixed train, validation, and test partitions

TemporalSplit uses inclusive index boundaries and creates the default scheme, enabling .train, .validation, and .test views.

train_end = dataset.index[int(len(dataset) * 0.60) - 1]
validation_end = dataset.index[int(len(dataset) * 0.80) - 1]

fixed = dataset.split(
    q.dataset.TemporalSplit(
        train_end=train_end,
        validation_end=validation_end,
    )
)

q.dataset.split_diagnostics(fixed, "default")
role rows proportion start end
partition
train fit 297 0.6 2024-07-25 2025-09-30
validation evaluate 99 0.2 2025-10-01 2026-02-23
test holdout 99 0.2 2026-02-24 2026-07-16
excluded excluded 0 0.0 NaT NaT

Expanding walk-forward folds

q.dataset.TimeSeriesSplit wraps sklearn.model_selection.TimeSeriesSplit. With no max_train_size, each fit window expands. A five-session gap separates training from each 42-session test window.

expanding_splitter = q.dataset.TimeSeriesSplit(
    n_splits=4,
    test_size=42,
    gap=5,
    name="expanding",
)
expanding = fixed.split(expanding_splitter)

print(expanding.splits["expanding"].metadata["backend"])
q.dataset.split_diagnostics(expanding, "expanding", "fold_1")
sklearn.model_selection.TimeSeriesSplit
role rows proportion start end
partition
train fit 322 0.650505 2024-07-25 2025-11-04
test holdout 42 0.084848 2025-11-12 2026-01-13
excluded excluded 131 0.264646 2025-11-05 2026-07-16

Rolling walk-forward folds

Setting sklearn’s max_train_size turns the same wrapper into a rolling-window splitter. Here every fold fits on at most 252 SPY sessions, approximately one trading year.

rolling_splitter = q.dataset.TimeSeriesSplit(
    n_splits=4,
    test_size=42,
    gap=5,
    max_train_size=252,
    name="rolling",
)
rolling = expanding.split(rolling_splitter)

q.dataset.split_diagnostics(rolling, "rolling", "fold_4")
role rows proportion start end
partition
train fit 252 0.509091 2025-05-07 2026-05-07
test holdout 42 0.084848 2026-05-15 2026-07-16
excluded excluded 201 0.406061 2024-07-25 2026-05-14

Purging and embargo

PurgedTimeSeriesSplit begins with sklearn’s walk-forward folds, removes fit rows whose five-session label horizon reaches the test start, and then applies an embargo. Integer embargoes count rows, which are trading sessions for this SPY index; timedelta strings are also supported.

purged_splitter = q.dataset.PurgedTimeSeriesSplit(
    n_splits=4,
    test_size=42,
    max_train_size=252,
    label_end="label_end_time",
    embargo=2,
)
split_dataset = rolling.split(purged_splitter)

calendar_embargo = q.dataset.PurgedTimeSeriesSplit(
    n_splits=4,
    test_size=42,
    max_train_size=252,
    label_end="label_end_time",
    embargo="7D",
    name="calendar_embargo",
)
split_dataset = split_dataset.split(calendar_embargo)

display(q.dataset.split_diagnostics(split_dataset, "purged_walk_forward", "fold_1"))
display(q.dataset.split_diagnostics(split_dataset, "calendar_embargo", "fold_1"))
q.dataset.audit_splits(split_dataset, "purged_walk_forward")
role rows proportion start end
partition
train fit 245 0.494949 2024-11-08 2025-10-31
test holdout 42 0.084848 2025-11-12 2026-01-13
excluded excluded 208 0.420202 2024-07-25 2026-07-16
role rows proportion start end
partition
train fit 247 0.498990 2024-11-08 2025-11-04
test holdout 42 0.084848 2025-11-12 2026-01-13
excluded excluded 206 0.416162 2024-07-25 2026-07-16
fit_rows evaluation_rows temporally_ordered label_horizons_clear passed
fold
fold_1 245 42 True True True
fold_2 245 42 True True True
fold_3 245 42 True True True
fold_4 245 42 True True True

Fit transformations without leakage and persist the result

q.transform.Pipeline wraps sklearn’s pipeline but selects fit rows from QRT partition roles. Iterating or indexing a dataset’s split scheme returns a bound Fold, so the pipeline can infer its scheme and fold without extra arguments. It transforms every row with the learned state and returns a fold bound to the transformed parent dataset. Dataset persistence retains all five split schemes and aligned components.

pipeline = q.transform.Pipeline([("scale", StandardScaler())])
fold = split_dataset.splits["purged_walk_forward"]["fold_1"]
processed_fold = pipeline.fit_transform(fold)
processed = processed_fold.dataset

fit_view = processed_fold.train
assert fit_view.X.mean().abs().max() < 1e-12
assert processed.y.index.equals(split_dataset.y.index)

with TemporaryDirectory() as directory:
    path = f"{directory}/spy-dataset.joblib"
    processed.save(path)
    restored = q.dataset.Dataset.load(path)
    pd.testing.assert_frame_equal(restored.X, processed.X)
    assert restored.splits.keys() == processed.splits.keys()

pd.Series(pipeline.fit_provenance_, name="fit provenance")
scheme        purged_walk_forward
fold                       fold_1
partitions               (train,)
rows                          245
start         2024-11-08 00:00:00
end           2025-10-31 00:00:00
Name: fit provenance, dtype: object

Compare and validate every solution

Each fold keeps complete membership by assigning unused rows to excluded. The final checks verify coverage, disjoint partition counts, sklearn provenance, and leakage-free purged folds.

examples = [
    ("default", "default"),
    ("expanding", "fold_1"),
    ("rolling", "fold_1"),
    ("purged_walk_forward", "fold_1"),
    ("calendar_embargo", "fold_1"),
]

summaries = []
for scheme, fold in examples:
    diagnostics = q.dataset.split_diagnostics(split_dataset, scheme, fold)
    assert diagnostics["rows"].sum() == len(split_dataset)
    summaries.append(
        diagnostics.reset_index().assign(scheme=scheme, fold=fold)
    )

assert split_dataset.splits["expanding"].metadata["backend"] == "sklearn.model_selection.TimeSeriesSplit"
assert q.dataset.audit_splits(split_dataset, "purged_walk_forward")["passed"].all()
assert q.dataset.audit_splits(split_dataset, "calendar_embargo")["passed"].all()

pd.concat(summaries, ignore_index=True)[
    ["scheme", "fold", "partition", "role", "rows", "proportion", "start", "end"]
]
scheme fold partition role rows proportion start end
0 default default train fit 297 0.600000 2024-07-25 2025-09-30
1 default default validation evaluate 99 0.200000 2025-10-01 2026-02-23
2 default default test holdout 99 0.200000 2026-02-24 2026-07-16
3 default default excluded excluded 0 0.000000 NaT NaT
4 expanding fold_1 train fit 322 0.650505 2024-07-25 2025-11-04
5 expanding fold_1 test holdout 42 0.084848 2025-11-12 2026-01-13
6 expanding fold_1 excluded excluded 131 0.264646 2025-11-05 2026-07-16
7 rolling fold_1 train fit 252 0.509091 2024-11-01 2025-11-04
8 rolling fold_1 test holdout 42 0.084848 2025-11-12 2026-01-13
9 rolling fold_1 excluded excluded 201 0.406061 2024-07-25 2026-07-16
10 purged_walk_forward fold_1 train fit 245 0.494949 2024-11-08 2025-10-31
11 purged_walk_forward fold_1 test holdout 42 0.084848 2025-11-12 2026-01-13
12 purged_walk_forward fold_1 excluded excluded 208 0.420202 2024-07-25 2026-07-16
13 calendar_embargo fold_1 train fit 247 0.498990 2024-11-08 2025-11-04
14 calendar_embargo fold_1 test holdout 42 0.084848 2025-11-12 2026-01-13
15 calendar_embargo fold_1 excluded excluded 206 0.416162 2024-07-25 2026-07-16
Back to top