vtools.data package

Submodules

vtools.data.dst module

Daylight Savings Time Conversion

This module provides the function dst_st() for converting a pandas Series/Dataframe with a naive DatetimeIndex that observes daylight savings time (DST) to a fixed standard time zone (e.g., PST) using POSIX conventions.

See the automatic API documentation for details: vtools.data.dst.dst_st()

dst_st(ts, src_tz: str = 'US/Pacific', target_tz: str = 'Etc/GMT+8')

Convert a pandas Series with a datetime index from a timezone-unaware index that observes DST (e.g., US/Pacific) to a fixed standard time zone (e.g., Etc/GMT+8) using POSIX conventions.

Parameters:

tspandas.Series

Time series with a naive (timezone-unaware) DatetimeIndex.

src_tzstr, optional

Source timezone name (default is ‘US/Pacific’).

target_tzstr, optional

Target standard timezone name (default is ‘Etc/GMT+8’).

Returns:

pandas.Series

Time series with index converted to the target standard timezone and made naive.

Notes

• The function assumes the index is not already timezone-aware.

• ‘Etc/GMT+8’ is the correct tz name for UTC-8 (PST) in pytz; note the sign is reversed from what might be expected.

• Handles ambiguous/nonexistent times due to DST transitions.

• The returned index is naive (timezone-unaware) but represents the correct standard time.

• If the input index is already timezone-aware, this function will raise an error.

Examples

>>> import pandas as pd
>>> from vtools import dst_st
>>> rng = pd.date_range("2023-11-05 00:00", "2023-11-05 04:00", freq="30min")
>>> ts = pd.Series(range(len(rng)), index=rng)
>>> converted = dst_st(ts)
>>> print(converted)
2023-11-05 00:00:00    0
2023-11-05 00:30:00    1
2023-11-05 01:00:00    2
2023-11-05 01:30:00    3
2023-11-05 02:30:00    5
2023-11-05 03:00:00    6
2023-11-05 03:30:00    7
2023-11-05 04:00:00    8
dtype: int64

vtools.data.duplicate_index module

inspect_duplicate_index(ts, label=None, max_times=5, max_rows=5)

Print a concise diagnostic for duplicate timestamps in a time series.

Parameters:

tspandas.DataFrame or Series

Time series with DatetimeIndex.

labelstr, optional

Context label (e.g., filename or pattern).

max_timesint

Number of duplicate timestamps to report.

max_rowsint

Number of rows per timestamp to print.

vtools.data.gap module

class Enum(value)

Bases: object

Generic enumeration.

Derive from this class to define new enumerations.

__dict__ = mappingproxy({'__module__': 'enum', '__doc__': '\n    Generic enumeration.\n\n    Derive from this class to define new enumerations.\n    ', '__new__': <staticmethod(<function Enum.__new__>)>, '_generate_next_value_': <function Enum._generate_next_value_>, '_missing_': <classmethod(<function Enum._missing_>)>, '__repr__': <function Enum.__repr__>, '__str__': <function Enum.__str__>, '__dir__': <function Enum.__dir__>, '__format__': <function Enum.__format__>, '__hash__': <function Enum.__hash__>, '__reduce_ex__': <function Enum.__reduce_ex__>, 'name': <types.DynamicClassAttribute object>, 'value': <types.DynamicClassAttribute object>, '__dict__': <attribute '__dict__' of 'Enum' objects>, '__weakref__': <attribute '__weakref__' of 'Enum' objects>, '_member_names_': [], '_member_map_': {}, '_member_type_': <class 'object'>, '_value2member_map_': {}, '__annotations__': {}})

__doc__ = '\n    Generic enumeration.\n\n    Derive from this class to define new enumerations.\n    '

__module__ = 'enum'

__weakref__

list of weak references to the object (if defined)

_generate_next_value_(start, count, last_values)

Generate the next value when not given.

name: the name of the member start: the initial start value or None count: the number of existing members last_value: the last value assigned or None

_member_map_ = {}

_member_names_ = []

_member_type_

alias of object

classmethod _missing_(value)

_value2member_map_ = {}

name

The name of the Enum member.

value

The value of the Enum member.

class GapSpec(n_gaps: int = 60, min_len: int = 70, max_len: int = 900, seed: Optional[int] = 123, strategy: vtools.data.gap.GapStrategy = <GapStrategy.TARGET_ONLY: 1>, ensure_room: int = 2)

Bases: object

__annotations__ = {'ensure_room': <class 'int'>, 'max_len': <class 'int'>, 'min_len': <class 'int'>, 'n_gaps': <class 'int'>, 'seed': typing.Optional[int], 'strategy': <enum 'GapStrategy'>}

__dataclass_fields__ = {'ensure_room': Field(name='ensure_room',type=<class 'int'>,default=2,default_factory=<dataclasses._MISSING_TYPE object>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'max_len': Field(name='max_len',type=<class 'int'>,default=900,default_factory=<dataclasses._MISSING_TYPE object>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'min_len': Field(name='min_len',type=<class 'int'>,default=70,default_factory=<dataclasses._MISSING_TYPE object>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'n_gaps': Field(name='n_gaps',type=<class 'int'>,default=60,default_factory=<dataclasses._MISSING_TYPE object>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'seed': Field(name='seed',type=typing.Optional[int],default=123,default_factory=<dataclasses._MISSING_TYPE object>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'strategy': Field(name='strategy',type=<enum 'GapStrategy'>,default=<GapStrategy.TARGET_ONLY: 1>,default_factory=<dataclasses._MISSING_TYPE object>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD)}

__dataclass_params__ = _DataclassParams(init=True,repr=True,eq=True,order=False,unsafe_hash=False,frozen=False)

__dict__ = mappingproxy({'__module__': 'vtools.data.gap', '__annotations__': {'n_gaps': <class 'int'>, 'min_len': <class 'int'>, 'max_len': <class 'int'>, 'seed': typing.Optional[int], 'strategy': <enum 'GapStrategy'>, 'ensure_room': <class 'int'>}, 'n_gaps': 60, 'min_len': 70, 'max_len': 900, 'seed': 123, 'strategy': <GapStrategy.TARGET_ONLY: 1>, 'ensure_room': 2, '__dict__': <attribute '__dict__' of 'GapSpec' objects>, '__weakref__': <attribute '__weakref__' of 'GapSpec' objects>, '__doc__': 'GapSpec(n_gaps: int = 60, min_len: int = 70, max_len: int = 900, seed: Optional[int] = 123, strategy: vtools.data.gap.GapStrategy = <GapStrategy.TARGET_ONLY: 1>, ensure_room: int = 2)', '__dataclass_params__': _DataclassParams(init=True,repr=True,eq=True,order=False,unsafe_hash=False,frozen=False), '__dataclass_fields__': {'n_gaps': Field(name='n_gaps',type=<class 'int'>,default=60,default_factory=<dataclasses._MISSING_TYPE object>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'min_len': Field(name='min_len',type=<class 'int'>,default=70,default_factory=<dataclasses._MISSING_TYPE object>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'max_len': Field(name='max_len',type=<class 'int'>,default=900,default_factory=<dataclasses._MISSING_TYPE object>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'seed': Field(name='seed',type=typing.Optional[int],default=123,default_factory=<dataclasses._MISSING_TYPE object>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'strategy': Field(name='strategy',type=<enum 'GapStrategy'>,default=<GapStrategy.TARGET_ONLY: 1>,default_factory=<dataclasses._MISSING_TYPE object>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD), 'ensure_room': Field(name='ensure_room',type=<class 'int'>,default=2,default_factory=<dataclasses._MISSING_TYPE object>,init=True,repr=True,hash=None,compare=True,metadata=mappingproxy({}),kw_only=False,_field_type=_FIELD)}, '__init__': <function GapSpec.__init__>, '__repr__': <function GapSpec.__repr__>, '__eq__': <function GapSpec.__eq__>, '__hash__': None, '__match_args__': ('n_gaps', 'min_len', 'max_len', 'seed', 'strategy', 'ensure_room')})

__doc__ = 'GapSpec(n_gaps: int = 60, min_len: int = 70, max_len: int = 900, seed: Optional[int] = 123, strategy: vtools.data.gap.GapStrategy = <GapStrategy.TARGET_ONLY: 1>, ensure_room: int = 2)'

__eq__(other)

Return self==value.

__hash__ = None

__init__(n_gaps: int = 60, min_len: int = 70, max_len: int = 900, seed: int | None = 123, strategy: GapStrategy = GapStrategy.TARGET_ONLY, ensure_room: int = 2) → None

__match_args__ = ('n_gaps', 'min_len', 'max_len', 'seed', 'strategy', 'ensure_room')

__module__ = 'vtools.data.gap'

__repr__()

Return repr(self).

__weakref__

list of weak references to the object (if defined)

ensure_room: int = 2

max_len: int = 900

min_len: int = 70

n_gaps: int = 60

seed: int | None = 123

strategy: GapStrategy = 1

class GapStrategy(value)

Bases: Enum

Where to create synthetic gaps.

BOTH = 2

STAGGERED = 3

TARGET_ONLY = 1

__doc__ = 'Where to create synthetic gaps.'

__module__ = 'vtools.data.gap'

_choose_gap_windows(n: int, spec: GapSpec) → List[Tuple[int, int]]

apply_gaps(target: Series, neighbor: Series | DataFrame, spec: GapSpec) → Tuple[Series, Series | DataFrame, Dict[str, List[Tuple[Timestamp, Timestamp]]]]

Apply synthetic gaps to target and/or neighbor.

Parameters:

target, neighbortime-aligned inputs

specGapSpec

Returns:

(target_gapped, neighbor_gapped, gap_windows)

• gap_windows provides timestamp ranges actually masked under keys ‘target’ and possibly ‘neighbor’.

class auto

Bases: object

Instances are replaced with an appropriate value in Enum class suites.

__dict__ = mappingproxy({'__module__': 'enum', '__doc__': '\n    Instances are replaced with an appropriate value in Enum class suites.\n    ', 'value': <object object>, '__dict__': <attribute '__dict__' of 'auto' objects>, '__weakref__': <attribute '__weakref__' of 'auto' objects>, '__annotations__': {}})

__doc__ = '\n    Instances are replaced with an appropriate value in Enum class suites.\n    '

__module__ = 'enum'

__weakref__

list of weak references to the object (if defined)

value = <object object>

dataclass(cls=None, /, *, init=True, repr=True, eq=True, order=False, unsafe_hash=False, frozen=False, match_args=True, kw_only=False, slots=False)

Returns the same class as was passed in, with dunder methods added based on the fields defined in the class.

Examines PEP 526 __annotations__ to determine fields.

If init is true, an __init__() method is added to the class. If repr is true, a __repr__() method is added. If order is true, rich comparison dunder methods are added. If unsafe_hash is true, a __hash__() method function is added. If frozen is true, fields may not be assigned to after instance creation. If match_args is true, the __match_args__ tuple is added. If kw_only is true, then by default all fields are keyword-only. If slots is true, an __slots__ attribute is added.

describe_null(dset, name, context=2)

If dset is a DataFrame, run describe_series_gaps on each column. If it’s a Series, just run it once.

describe_series_gaps(s: Series, name: str, context: int = 2)

Print gaps in a single Series s, showing context non-null points before and after each gap, with an ellipsis marker in between.

example_gap()

gap_count(ts, state='gap', dtype=<class 'int'>)

Count missing data Identifies gaps (runs of missing or non-missing data) and quantifies the length of the gap in terms of number of samples, which works better for regular series. Each time point receives the length of the run.

Parameters:

tsDataFrame

Time series to analyze

statestr one of ‘gap’|’good’|’both’

State to count. If state is gap, block size of missing data are counted and reported for time points in the gap (every point in a given gap will receive the same value). Non missing data will have a size of zero. Setting state to ‘good’ inverts this – missing blocks are reported as zero and good data are counted.

dtypestr or type

Data type of output, should be acceptable to pandas astype

gap_distance(ts, disttype='count', to='good')

For each element of ts, count the distance to the nearest good data/or bad data.

Parameters:

tsDataFrame

Time series to analyze

disttypestr one of ‘bad’|’good’

If disttype = “count” this is the number of values. If dist_type=”freq” it is in the units of ts.freq

(so if freq == “15min” it is in minutes”)

tostr one of ‘bad’|’good’

If to = “good” this is the distance to the nearest good data (which is 0 for good data).

If to = “bad”, this is the distance to the nearest nan (which is 0 for nan).

Returns:

resultDataFrame

A new regular time series with the same freq as the argument holding the distance of good/bad data.

gap_size(ts)

Identifies gaps (runs of missing data) and quantifies the length of the gap. Each time point receives the length of the run in terms of seconds or number of values in the time dimension, with non-missing data returning zero. Time is measured from the time the data first started being missing to when the data first starts being not missing .

Parameters:

tsDataFrame

Returns:

resultDataFrame

A new regular time series with the same freq as the argument holding the size of the gap.

Examples

>>> ndx = pd.date_range(pd.Timestamp(2017,1,1,12),freq='15min',periods=10)
>>> vals0 = np.arange(0.,10.,dtype='d')
>>> vals1 = np.arange(0.,10.,dtype='d')
>>> vals2 =  np.arange(0.,10.,dtype='d')
>>> vals0[0:3] = np.nan
>>> vals0[7:-1] = np.nan
>>> vals1[2:4] = np.nan>>>
>>> vals1[6] = np.nan
>>> vals1[9] = np.nan

>>> df = pd.DataFrame({'vals0':vals0,'vals1':vals1,'vals2':vals2},index = ndx)
>>> out = gap_size(df)
>>> print(df)
                         vals0  vals1  vals2
2017-01-01 12:00:00    NaN    0.0    0.0
2017-01-01 12:15:00    NaN    1.0    1.0
2017-01-01 12:30:00    NaN    NaN    2.0
2017-01-01 12:45:00    3.0    NaN    3.0
2017-01-01 13:00:00    4.0    4.0    4.0
2017-01-01 13:15:00    5.0    5.0    5.0
2017-01-01 13:30:00    6.0    NaN    6.0
2017-01-01 13:45:00    NaN    7.0    7.0
2017-01-01 14:00:00    NaN    8.0    8.0
2017-01-01 14:15:00    9.0    NaN    9.0
>>> print(out)
                         vals0  vals1  vals2
2017-01-01 12:00:00   45.0    0.0    0.0
2017-01-01 12:15:00   45.0    0.0    0.0
2017-01-01 12:30:00   45.0   30.0    0.0
2017-01-01 12:45:00    0.0   30.0    0.0
2017-01-01 13:00:00    0.0    0.0    0.0
2017-01-01 13:15:00    0.0    0.0    0.0
2017-01-01 13:30:00    0.0   15.0    0.0
2017-01-01 13:45:00   30.0    0.0    0.0
2017-01-01 14:00:00   30.0    0.0    0.0
2017-01-01 14:15:00    0.0    0.0    0.0

vtools.data.indexing module

__all__ = ['resolve_common_freq', 'regular_index_from_valid_extent', 'reindex_to_continuous', 'infer_freq_robust', 'inferred_regular_freq', 'compare_regular_freq']

Index construction and frequency-handling utilities for time series operations.

This module provides low-level helpers for: - enforcing frequency consistency across multiple time series, - constructing regular time indexes from valid data extents, and - reindexing existing series onto continuous grids when possible.

These functions are used by higher-level operations such as merging, splicing, and blending of time series.

compare_regular_freq(ts_src, ts_dst)

Compare source and destination time series for frequency compatibility.

This function supports reconcile-policy decisions by checking whether two already-read time series are regular and, if so, whether their inferred frequencies match.

Parameters:

ts_srcpandas.Series or pandas.DataFrame

Source (staged) time series.

ts_dstpandas.Series or pandas.DataFrame

Destination (repo) time series.

Returns:

statusstr

Compatibility classification. One of: - "both_regular_same" - "both_regular_different" - "src_irregular" - "dst_irregular" - "both_irregular"

reasonstr

Human-readable explanation of the classification.

src_freqpandas offset or None

Inferred source frequency, if regular.

dst_freqpandas offset or None

Inferred destination frequency, if regular.

See also

inferred_regular_freq

Check whether a single time series is regular and determine its inferred frequency.

Notes

This helper is asymmetric only in naming: src and dst are reported separately because reconcile policy may differ depending on which side is irregular.

It does not modify the inputs and does not attempt to reconcile or repair them.

infer_freq_robust(index, preferred=['h', '15min', '6min', '10min', 'h', 'd'], **kwargs)

Infer the frequency of a time series index using multiple strategies.

Attempts to detect the frequency of a DatetimeIndex by trying various approaches: direct inference on the full index, on subsets at different positions, and finally by testing against a list of preferred frequencies. This robust approach handles indices with irregular intervals or gaps better than pandas’ native infer_freq.

Parameters:

indexpd.DatetimeIndex

The time series index to analyze.

preferredlist of str, optional

Preferred frequency strings to test as fallback (default: [“h”, “15min”, “6min”, “10min”, “h”, “d”]). Each string should be a valid pandas frequency alias.

**kwargs

Additional keyword arguments (currently unused, reserved for future use).

Returns:

str

A pandas frequency alias string (e.g., “h”, “D”, “15min”).

Raises:

ValueError

If all frequency inference attempts fail.

Notes

• Index is rounded to 1-minute resolution before analysis.

• For short indices (< 8 points), direct inference is attempted.

• For longer indices, inference is tried on the last 7 points, then at 75% and 50% positions, then at the start.

• Finally, preferred frequencies are tested by checking if at least 98% of index values round correctly to that frequency.

inferred_regular_freq(ts)

Determine whether a time series has a regular inferred frequency.

This function is a strict checker for already-read time series. It does not coerce, round, or repair timestamps. It inspects the existing index and reports whether a regular frequency can be established.

Parameters:

tspandas.Series or pandas.DataFrame

Time series with a DatetimeIndex or PeriodIndex.

Returns:

freqpandas offset or None

The inferred or attached frequency if the series is regular. Returns None if the series is not regular.

reasonstr

Diagnostic reason code. One of: - "ok" - "degenerate" - "not_datetime_like" - "not_monotonic" - "duplicates" - "infer_failed"

See also

compare_regular_freq

Compare the inferred regular frequencies of two time series for reconcile compatibility.

Notes

This checker is intentionally strict and non-repairing. It is intended for validation and reconcile policy, not ingestion recovery.

Frequency is determined by: 1. Using index.freq if present. 2. Otherwise trying pandas.infer_freq(index).

regular_index_from_valid_extent(series, freq)

Construct a regular index spanning the valid data extent of input series.

This function creates a regular (fixed-frequency) index that spans from the earliest first valid timestamp to the latest last valid timestamp across a collection of time series.

Parameters:

seriessequence of pandas.Series or pandas.DataFrame

Input time series objects. Each must have an index of the same type (DatetimeIndex or PeriodIndex). The index values are used to determine the overall time extent of valid data.

freqpandas offset

Frequency to use when constructing the regular index (e.g., pandas offset such as Hour, Day, etc.). Typically obtained from resolve_common_freq.

Returns:

indexpandas.DatetimeIndex or pandas.PeriodIndex

A regular index spanning from the minimum first valid timestamp to the maximum last valid timestamp across all input series.

If no valid timestamps are found in any series, an empty index of the same type as the first input is returned.

Raises:

ValueError

If the index type is not supported (i.e., not DatetimeIndex or PeriodIndex).

See also

resolve_common_freq

Determine whether a shared frequency exists across input indexes.

Notes

• The function uses first_valid_index() and last_valid_index() for each series, so leading and trailing NaNs are ignored when determining the time extent.

• The returned index includes both endpoints.

• No validation is performed to ensure that input series conform to the specified frequency.

reindex_to_continuous(result, freq)

Reindex a time series onto a regular grid if possible.

This function attempts to map an existing time series onto a continuous, fixed-frequency index spanning its full time extent. If the existing timestamps are not compatible with the target regular grid, the input is returned unchanged.

Parameters:

resultpandas.Series or pandas.DataFrame

Time series to be reindexed. Must have a DatetimeIndex or PeriodIndex.

freqpandas offset or None

Target frequency for the regular index. If None, no action is taken and result is returned unchanged.

Returns:

outpandas.Series or pandas.DataFrame

Reindexed time series if all existing timestamps align with the target regular grid. Otherwise, the original input is returned unchanged.

See also

regular_index_from_valid_extent

Construct a regular index prior to composition operations.

resolve_common_freq

Determine if a shared frequency can be enforced.

Notes

• The function first constructs a regular index from the minimum to maximum timestamps of result using the provided freq.

• If any existing timestamps are not present in the constructed regular index, the function does not reindex and instead clears the .freq attribute (if possible) before returning the original data.

• This behavior is intentionally conservative to avoid silently dropping or shifting data.

• When reindexing succeeds: - Missing timestamps are filled with NaN. - The .freq attribute is set on the resulting index if possible.

resolve_common_freq(indexes, preserve_freq=True)

Determine a common frequency across a collection of pandas indexes.

This function inspects the .freq attribute of each index and, if frequency preservation is requested, verifies that all non-null frequency attributes are identical. If so, that frequency is returned. Otherwise, an error is raised.

Parameters:

indexessequence of pandas.Index

Sequence of pandas Index objects (typically DatetimeIndex or PeriodIndex). Each index may or may not have a .freq attribute.

preserve_freqbool, default True

If True, enforce that all indexes with a non-null .freq attribute have identical frequencies. If any mismatch is detected, a ValueError is raised.

If False, no checking is performed and the function always returns None.

Returns:

freqpandas offset or None

The common frequency if one can be determined and preserve_freq=True. Returns None if: - preserve_freq=False, or - no index has a non-null .freq attribute.

Raises:

ValueError

If preserve_freq=True and multiple indexes define conflicting .freq attributes.

See also

regular_index_from_valid_extent

Construct a regular index once a common frequency has been established.

Notes

• This function relies only on the .freq attribute and does not attempt to infer frequency using pandas.infer_freq or the more robust vtools functions for that purpose.

• It is intended for use in routines that require strict consistency of sampling intervals across inputs.

to_timedelta(x)

Convert x to pandas.Timedelta if and only if it represents a fixed-length duration.

Notes

Numeric values are rejected because they are unit-ambiguous.

vtools.data.sample_series module

bessel_df()

Sample series with bessel function signals

extra()

interval(ts)

Sampling interval of series

jay_flinchem_chirptest(c1=3.5, c2=5.5, c3=0.0002, c4=6.75)

Approximation of the signal from Jay and Flinchem 1999 A comparison of methods for analysis of tidal records containing multi-scale non-tidal background energy that has a small tide with noisy, river-influenced amplitude and subtide

small_subtide(subtide_scale=0.0, add_nan=False)

Inspired by large tidal flow with small Qr undercurrent with 72hr period This is a tough lowpass filtering job because the diurnal band is large and must be supressed in order to see the more subtle subtidal amplitude

vtools.data.timeseries module

Time series module Helpers for creating regular and irregular time series, transforming irregular to regular and analyzing gaps.

datetime_elapsed(index_or_ts, reftime=None, dtype='d', inplace=False)

Convert a time series or DatetimeIndex to an integer/double series of elapsed time

Parameters:

index_or_tsDatatimeIndex

Time series or index to be transformed

reftimeDatatimeIndex or something convertible

The reference time upon which elapsed time is measured. Default of None means start of series

dtypestr like ‘i’ or ‘d’ or type like int (Int64) or float (Float64)

Data type for output, which starts out as a Float64 (‘d’) and gets converted, typically to Int64 (‘i’)

inplacebool

If input is a data frame, replaces the index in-place with no copy

Returns:

result

A new index using elapsed time from reftime as its value and of type dtype

days(d)

Create a time interval representing d days

divide_interval(a, b, *, tol=1e-12, require_int=True)

Divide two intervals (or two scalars) safely.

• interval / interval -> float ratio (or int if require_int)

• scalar / scalar -> numeric ratio

• mixed scalar/interval -> TypeError

elapsed_datetime(index_or_ts, reftime=None, time_unit='s', inplace=False)

Convert a time series or numerical Index to a Datetime index or series

Parameters:

index_or_tsDatatimeIndex

Time series or index to be transformed with index in elapsed seconds from reftime

reftimeDatatimeIndex or something convertible

The reference time upon which datetimes are to be evaluated.

inplacebool

If input is a data frame, replaces the index in-place with no copy

Returns:

result

A new index using DatetimeIndex inferred from elapsed time from reftime as its value and of type dtype

example()

extrapolate_ts(ts, start=None, end=None, method='ffill', val=None)

Extend a regular time series to a new start and/or end using a specified extrapolation method.

Parameters:

tspandas.Series or pandas.DataFrame

The input time series with a DateTimeIndex and a regular frequency.

startdatetime-like, optional

The new starting time. If None, no extension is done before the existing data.

enddatetime-like, optional

The new ending time. If None, no extension is done after the existing data.

method{‘ffill’, ‘bfill’, ‘linear_slope’, ‘taper’, ‘constant’}, default ‘ffill’

The method used to fill new values outside the original time range:

• ‘ffill’ : Forward-fill after the original data using its last value.

• ‘bfill’ : Backward-fill before the original data using its first value.

• ‘linear_slope’ : Bidirectional linear extrapolation using the first/last two points.

• ‘taper’ : One-sided linear interpolation to/from a specified value (val).

• ‘constant’ : One-sided constant value fill with val.

valfloat, optional

Required for ‘taper’ and ‘constant’. Specifies the value to use.

Returns:

extendedpandas.Series or pandas.DataFrame

The time series extended and filled using the selected method.

Raises:

ValueError

• If extrapolation rules are violated based on the method.

• If method requires or forbids val and it’s misused.

• If frequency cannot be inferred.

hours(h)

Create a time interval representing h hours

is_regular(ts, raise_exception=False)

Check if a pandas DataFrame, Series, or xarray object with a time axis (axis 0) has a regular index.

Regular means:

• The index is unique.

• For DatetimeIndex: equals a date_range with the inferred frequency.

• For numeric index (int/float): differences between consecutive values are constant.

Parameters:

tsDataFrame, Series, or xarray object.

Series to evaluate

raise_exceptionbool

If True, raises a ValueError when the index is not regular. Otherwise, returns False.

Returns:

bool : True if the index is regular; False otherwise.

minutes(m)

Create a time interval representing m minutes

months(m)

Create a time interval representing m months

numtype

alias of Number

rename_columns(ts: Series | DataFrame, colnames: str | Sequence[str] | Mapping[str, str] | Callable[[str], str], convert_df: bool = True) → Series | DataFrame

Rename columns (for DataFrame) or the name (for Series).

Parameters:

ts

pandas Series or DataFrame to rename.

colnames

• str

  • Series: set Series.name to this value.

  • DataFrame: treated as a single target name; the DataFrame must have exactly one column. Raises if there are multiple columns.

• Sequence[str]

  • Series: must be length 1; that single value becomes Series.name.

  • DataFrame: length must equal the number of columns; these become the new column names in order.

• Mapping[str, str]

  • DataFrame: passed to DataFrame.rename(columns=…).

  • Series: if the current Series.name is a key, it is mapped to the value. If not present, the name is left unchanged. Mapping with None can be used to rename a nameless series.

• Callable[[str], str]

  • DataFrame: applied to each column name (via rename).

  • Series: called with the current Series.name to compute the new name.

convert_dfbool, default True

If True and ts is a Series, convert it to a single-column DataFrame before renaming.

Returns:

pandas Series or DataFrame

A copy of ts with updated name(s).

Raises:

TypeError

If ts is not a Series or DataFrame, or colnames has an unsupported type.

ValueError

If a provided list length does not match the number of columns (DataFrame) or is not exactly 1 (Series), or if a single string is given for a multi-column DataFrame.

rts(data, start, freq, columns=None, props=None)

Create a regular or calendar time series from data and time parameters

Parameters:

dataarray_like

Should be a array/list of values. There is no restriction on data

type, but not all functionality like addition or interpolation will work on all data.

startPandas.Timestamp

Timestamp or a string or type that can be coerced to one.

interval_time_interval

Can also be a string representing a pandas freq.

Returns:

resultPandas.DataFrame

A regular time series with the freq attribute set

rts_formula(start, end, freq, valfunc=nan)

Create a regular time series filled with constant value or formula based on elapsed seconds

Parameters:

startPandas.Timestamp

Starting Timestamp or a string or type that can be coerced to one.

endPandas.Timestamp

Ending Timestamp or a string or type that can be coerced to one.

freq_time_interval

Can also be a string representing an interval.

valfuncdict

Constant or dictionary that maps column names to lambdas based on elapsed time from the starts of the series. An example would be {“value”: lambda x: np.nan}

Returns:

resultPandas.DataFrame

A regular time series with the freq attribute set

seconds(s)

Create a time interval representing s seconds

time_overlap(ts0, ts1, valid=True)

Check for overlapping time coverage between series Returns a tuple of start and end of overlapping periods. Only considers the time stamps of the start/end, possibly ignoring NaNs at the beginning if valid=True, does not check for actual time stamp alignment

to_dataframe(ts)

to_timedelta(x)

Convert x to pandas.Timedelta if and only if it represents a fixed-length duration.

Notes

Numeric values are rejected because they are unit-ambiguous.

years(y)

Create a time interval representing y years

vtools.data.vis_gap module

generate_sample_data()

interactive_gap_plot(df)

plot_missing_data(df, ax, min_gap_duration, overall_start, overall_end)

vtools.data.vtime module

Basic ops for creating, testing and manipulating times and time intervals. This module contains factory and helper functions for working with times and time intervals.

For time intervals (or deltas), VTools uses classes that are compatible with the “freq” argument of

requires a time and time interval system that is consistent (e.g. time+n*interval makes sense) and that can be applied to both calendar dependent and calendar-independent intervals. Because this requirement is not met by any one implementation it is recommended that you always use the factory functions in this module for creating intervals or testing whether an interval is valid.

compare_interval(a, b)

Compare two fixed-length intervals. Returns -1 if a < b, 0 if equal, 1 if a > b. Rejects calendar-dependent offsets like months/years.

days(d)

Create a time interval representing d days

divide_interval(a, b, *, tol=1e-12, require_int=True)

Divide two intervals (or two scalars) safely.

• interval / interval -> float ratio (or int if require_int)

• scalar / scalar -> numeric ratio

• mixed scalar/interval -> TypeError

dst_to_standard_naive(ts, dst_zone='US/Pacific', standard_zone='Etc/GMT+8')

Convert timezone-unaware series from a local (with daylight) time to standard time This would be useful, say, for converting a series that is PDT during summer to one that is not. The routine is mainly to treat cases where the time stamps at DST interfaces are not redundant – if they are you can probably use tz_convert and tz_localize with the ambiguous = ‘infer’ option and do the job more efficiently, but lots of databases don’t store data this way.

The choice of the standard_zone is, it seems, buggy. The defaults are supposed to convert from PST/PDT to pure PST, and the latter should be GMT-8. In a sense, this function is included before the behavior is really understood.

Only regular series are accepted … this is a quirk of the implementation

hours(h)

Create a time interval representing h hours

minutes(m)

Create a time interval representing m minutes

months(m)

Create a time interval representing m months

seconds(s)

Create a time interval representing s seconds

to_timedelta(x)

Convert x to pandas.Timedelta if and only if it represents a fixed-length duration.

Notes

Numeric values are rejected because they are unit-ambiguous.

years(y)

Create a time interval representing y years

Module contents