data.clean

data.clean

Deterministic cleaning and validation for market-data frames.

Functions

Name	Description
canonicalize_ohlcv	Canonicalize, order, deduplicate, and validate an OHLCV frame.
deduplicate	Return a copy with duplicate observations removed.
detect_gaps	Return one gap record per entity and pair of observed timestamps.
normalize_timestamps	Return a copy with a normalized datetime column or index.
validate_ohlcv	Validate canonical OHLCV columns, ordering, and market invariants.

canonicalize_ohlcv

data.clean.canonicalize_ohlcv(
    data,
    *,
    column_map=None,
    timestamp='datetime',
    entity_keys=('symbol',),
    timezone='UTC',
    duplicate_keep='last',
    allow_missing=False,
)

Canonicalize, order, deduplicate, and validate an OHLCV frame.

deduplicate

data.clean.deduplicate(data, *, subset=None, keep='last')

Return a copy with duplicate observations removed.

By default, observations are identified by symbol when present and by the datetime column or index.

detect_gaps

data.clean.detect_gaps(
    data,
    frequency,
    *,
    timestamp='datetime',
    entity_keys=('symbol',),
)

Return one gap record per entity and pair of observed timestamps.

Each record contains the entity keys, first and last missing timestamps, and number of missing periods at the requested regular frequency.

normalize_timestamps

data.clean.normalize_timestamps(
    data,
    *,
    timestamp='datetime',
    timezone='UTC',
    ambiguous='raise',
    nonexistent='raise',
)

Return a copy with a normalized datetime column or index.

Naive timestamps are localized to timezone and timezone-aware values are converted to it. Pass timezone=None to retain naive timestamps.

validate_ohlcv

data.clean.validate_ohlcv(
    data,
    *,
    timestamp='datetime',
    entity_keys=('symbol',),
    allow_missing=False,
)

Validate canonical OHLCV columns, ordering, and market invariants.