vtools.data package

Submodules

vtools.data.constants module

This module contains string constants used in timeseries properties.

AGGREGATION = 'aggregation'

Attribute name for sample aggregation over steps

MAX = 'MAX'

AGGREGATION value: Period max

MIN = 'MIN'

AGGREGATION value: Period min

MEAN = 'MEAN'

AGGREGATION value: Period mean

SUM = 'ACCUM'

AGGREGATION value: Accumulation

INDIVIDUAL = 'INST-VAL'

AGGREGATION value: Instantaneous values

TIMESTAMP = 'timestamp'

TIMESTAMP value: Attribute name for time stamping convention

PERIOD_START = 'PERIOD_START'

TIMESTAMP value: Stamped at beginning of period

PERIOD_END = 'PERIOD_END'

TIMESTAMP value: Stamped at end of period

vtools.data.sample_series module

arma(phi, theta, sigma, n, discard=0, verbose=0)

Generate a Gaussian ARMA(p,q) series. This was obtained from stack exchange and is not written in terms of a time series … however it can be used as an ingredient in a time series providing correlated random noise

Parameters:

phi : array

An array of length p with the AR coefficients (the AR part of the ARMA model).

theta : array

An array of length q with the MA coefficients (the MA part of the ARMA model).

sigma : float

Standard deviation of the Gaussian noise.

n : Length of the returned series.

discard: Number of data points that are going to be discarded (the higher

the better) to avoid dependence of the ARMA time-series on the initial values.

Returns:

Numpy array from an ARMA(p,q) process

create_sample_series()

example_data(name)

old_river_head_flow()

flow rate of old river at head with 15 minutes interval

pt_reyes_tidal_1hour_interval()

Sea surface level at Point Reyes from NOAA downsampled to 1hour interval from 11/01/2013-11/08/2013

pt_reyes_tidal_6min_interval()

Sea surface level at Point Reyes from NOAA with 6 min interval from 11/24/2013-11/25/2013

pt_reyes_tidal_with_gaps()

Sea surface level at Point Reys with gaps of different length”

synthetic_tide_series(days=7)

Return a week long synthetic tide

vtools.data.timeseries module

Time series module Module contains the TimeSeries class and the factory functions for creating regular and irregular time series, as well as some helper functions.

class TimeSeries(times, data, props, header=None)

Bases: object

Fundamental class for both regular and irregular time series. The preferred way to create a time series is with the rts or its factory functions, not with the constructor. The member attributes include the times and data in the series (which are paired) as well as methods of querying the time series properties (start, end, interval) of the time series.

centered(copy_data=False, neaten=True)

Return a time series with times centered between the timestamps of the original series.

Parameters:

copy_data : boolean,optional

If True, the result is an entirely new series with deep copy of all data and properties. Otherwise, it will share data and properties … more like a window than a new series.

neaten : boolean, optional

Returns:

result : TimeSeries

New series with shared or copied data with time centered between the timestamps of the original series. Note the new series has length one shorter than the original.

copy(start=None, end=None, left=False, right=False)

Perform a copy of this series, optionally with clipped start and end

Parameters:

start : datetime.datetime

Start time of copy

end : :py:class:`datetime.datetime`extr

End time of copy

left : boolean, optional

Overlap so that the copy catches the value previous to the given start time (useful for irregular)

right : boolean, optional

Overlap so that the copy catches the next value after the given end time (useful for irregular)

Returns:

result : TimeSeries

Copy of the series. No shared data with the original, so you can manipulate it without fear of overwriting the original series.

See also

window

Same concept but with shared data.

data

Data component of series (for all time)

end

Time at last element of series

has_gap()

find out if timeseries has nan data

index_after(tm)

return the indexes of the time series at/after points given by tm.

Input tm maybe a single datetime or list/array datetime, or single ticks or list/array of ticks. Accordingly, return will be a single index or array of indexes

If points are found, corresponding indexes will be return, if not, indexes whose time are directly after points will be returned.

Performance of this method is better if tm is given by ticks if this is convenient; otherwise, if given a sequence of times, the method will convert the sequence to ticks automatically.

index_before(tm)

Return the index of the time series at point tm

is_regular()

Returns true if the time series is regular A regular series is considered regular if it has a sampling interval. This interval may or may not be calendar dependent.

props

Dictionary containing attributes, metadata and user properties

replace(ts, start, end)

Return a time series with data replaced by input ts or constant

Parameters:

ts : TimeSeries or constant

Data or constant to be used for replacing

start : datetime.datetime

Start time of window

end : datetime.datetime

End time of window

Returns:

result : TimeSeries

A new time series.

shift(interval, start=None, end=None)

Inplace shift section of time series by a interval.

Parameters:

interval : dateutil.relativedelta, datetime.timedelta, string

start : datetime.datetime, string

end : datetime.datetime, string

Returns:

result : boolean

return true if succeeded.

start

Time at first element of series

ticks

Array of long integer ticks representing the time index of the series

times

Array of datetimes represented by series

ts_binary(f)

ts_inplace_binary(f)

ts_unary(f)

window(start=None, end=None, left=False, right=False)

Provide a shared-memory ts (shared data and props) with a reduced time window

Note

if left=True, the first time point will be at or before the given start time otherwise, the first time will be at or after the given start. same for right.

Parameters:

start : datetime.datetime

Start time of window

end : datetime.datetime

End time of window

left : boolean, optional

Overlap so that the copy catches the value previous to the given start time (useful for irregular)

right : boolean, optional

Overlap so that the copy catches the next value after the given end time (useful for irregular)

Returns:

result : TimeSeries

Copy of the series,shared data with the original.

See also

copy

Same concept but with copied data.

class TimeSeriesElement(time_data)

Bases: object

ticks

Time point in long integer ticks of element

time

Time at element

value

Value at element

extrapolate_ts(ts, start=None, end=None, method='constant', val=nan)

Extend a regular time series with newer start/end

Parameters:

start : datetime.datetime

The starting time to be extended to.Optional, default no extension.

end : datetime.datetime

The ending time to be extended to.Optional,default no extension.

method : :string

Method to fill the extended part of resulting series. either “taper” or”constant”. default constant.

val : float

Constant will be filled or the value that the last non-nan gets tapered to. Only constant is supported for multivariate data

Returns:

result : TimeSeries

An new time series extended to the input start and end. This function will only extend, not clip, so if you want to be sure the returned function is the correct siae you need to apply a window afterwards.

index_after(seq, tm)

Return the integer index of a time sequence that is on or after tm

Parameters:

seq : time_sequence

The sequence whose index will be searched

tm : datetime.datetime

Time whose index is sought

Returns:

The integer index in the sequence that fall on or after tm.

When tm bigger than the last item of seq, it will return len(seq).

index_before(seq, tm)

Return the integer index of a time sequence that is on or before tm

Parameters:

seq : time_sequence

The sequence whose index will be searched

tm : datetime.datetime

Time whose index is sought

Returns:

The integer index in the sequence that fall on or before tm.

indexes_after(seq, tm)

Return an array of indexes representing the index of seq that is on or after each

member of tm.

Parameters:

seq : time_sequence

The sequence whose index will be searched

tm : list of datetime.datetime

List of times whose index is sought

When tm is bigger than the last item of seq, it will return len(seq).

When tm is smaller than the first time of seq, it will return 0.

its(times, data, props=None)

Create an irregular time series from time and data sequences

Parameters:

times : time_sequence

An array or list of datetimes or ticks

data : array or list of values

An array/list of values. No restriction on data type, but not all functionality will work on every data type.

Returns:

result : TimeSeries

An irregular TimeSeries instance

its2rts(its, interval, original_dates=True)

Convert an irregular time series to a regular.

This function assumes observations were taken at “almost regular” intervals with some variation due to clocks/recording. It nudges the time to “neat” time points to obtain the corresponding regular index, allowing gaps. There is no correctness checking, The dates are stored at the original “imperfect” time points if original_dates == True, otherwise at the “nudged” regular times.

Parameters:

its : TimeSeries

A irregular time series.

interval : time_interval

Interval of resulting regular timeseries,Can also be a string representing an interval that is understood by vools.vtime.parse_interval().

original_dates:boolean,optional

Use original datetime or nudged regular times.

Returns:

result : TimeSeries

A regular time series.

prep_binary(ts1, ts2)

Create data for time-aligned op binary operation between two series Returns data holders and selections required to carry out index-aligned binary operations on two series. Mostly of interest to programmers.

Parameters:

ts1,ts2 : TimeSeries

Two regular time series with similar time intervals

Returns:

seq : time_sequence

Representing the union of times from the series

start : py:class:datetime.datetime

The start of the sequence as a datetime

slice0 : slice

Indexing slice within seq covered by ranges of both ts1 and ts2

slice1 : slice

Indexing slice within ts1 representing the region intersecting ts2

slice2 : slice

Indexing slice within ts2 representing the region intersecting ts1

range_intersect(ts0, ts1)

Intersection of time ranges of two series.

Note

may return result with union_start > union_end, in which case there is no intersection.

Parameters:

ts0,ts1 : TimeSeries

The series whose union is to be found

Returns:

range : (datetime.datetime, datetime.datetime)

A tuple representing the start and end of the intersect of time ranges of ts0 and ts1

range_union(ts0, ts1)

Union of time ranges of two series.

Parameters:

ts0,ts1 : TimeSeries

The series whose union is to be found

Returns:

range : (datetime.datetime, datetime.datetime)

A tuple representing the start and end of the union of time ranges of ts0 and ts1, as determined by the earliest start and the latest end.

rts(data, start, interval, props=None)

Create a regular or calendar time series from data and time parameters

Parameters:

data : array_like

Should be a array/list of values. There is no restriction on data

type, but not all functionality like addition or interpolation will work on all data.

start : datetime.datetime

Can also be a string representing a datetime.

interval : time_interval

Can also be a string representing an interval that is understood by vools.vtime.parse_interval().

Returns:

result : TimeSeries

A regular time series.

rts_constant(start, end, interval, val=nan)

Create a regular or calendar time series filled with constant value

Parameters:

start : datetime.datetime

Starting time, can also be a string representing a datetime.

end : datetime.datetime

Ending time,can also be a string representing a datetime.

interval : time_interval

Can also be a string representing an interval that is understood by vools.vtime.parse_interval().

val : float,int

Constant will be filled into the time series. Optional, default is nan.

Returns:

result : TimeSeries

A regular time series wiht constant values

vtools.data.vtime module

Basic ops for creating, testing and manipulating times and time intervals. This module contains factory and helper functions for working with times and time intervals.

The module will import the name datetime and is designed to work exclusively with python datetimes. In addition, datetimes are convertible to long integer timestamps called ticks. The resolution of 1 tick in ticks per second may be obtained using the resolution() function or certain utility constants such as ticks_per_day. Never code with hard wired numbers.

For time intervals (or deltas), VTools requires a time and time interval system that is consistent (e.g. time+n*interval makes sense) and that can be applied to both calendar dependent and calendar-independent intervals. Because this requirement is not met by any one implementation it is recommended that you always use the factory functions in this module for creating intervals or testing whether an interval is valid.

align(timepoint, interval, side)

Finds a neat time point that is calendar aligned with the given interval either to the right (+1) or the left (-1) of the arg timepoint. E.g., align(01Jan1992 00:

days(d)

Create a time interval representing d days

hours(h)

Create a time interval representing h hours

increment(time, interval, nintvl=1)

Increment a time by an interval. Safely increment time by interval ninterval times. This function helps resolve a bug in dateutil. Arguments:

time: time to be incremented interval: the interval to use nintvl: number of times to increment it

infer_interval(time_sequence, fraction, standard=None)

Infer a time interval from a time sequence using differences

Parameters:

time_sequence : time_sequence

sequence to be analyzed

fraction : float

fraction of intervals in the sample that must conform once times are

rounded to one minute. For instance, if four times are input,

this will create three differences, so a fraction=0.67 would require that two out

of three agree.

standard : list of standard intervals. If None, any non-calendar interval is accepted

as is an interval of one month. Otherwise, this argument should be a list and

a ValueError is generated if the interval isn’t on the list.

Returns:

interval : time_interval

time interval of appropriate class inferred from data

is_calendar_dependent(intvl)

Determine whether the input is a calendar dependent time interval. Returns true if the length of the interval does not depend on the calendar, currently implemented by determining whether the interval includes months or years.

is_interval(intvl)

Determine whether the argument is a time interval. This is the safe way to determine whether the input is a time interval class understood by Vtools.

minutes(m)

Create a time interval representing m minutes

months(m)

Create a time interval representing m months

number_intervals(start, end, dt)

Calculate the number of intervals dt between start and end time. The result is the max number_intervals such that

start + number_intervals*dt <= end.

parse_interval(interval_string)

Parse interval expressed as string and return a valid time interval

Parameters:

interval_string : string

A string representing the interval to be parsed. May contain spaces, numbers, and abbreviations:

• s,sec,second
• m,min,minute
• h,hr,hour
• d,day
• mon,month
• y,yr,year

Returns:

interval : time_interval

time interval of appropriate class for vtools depending on calendar dependence

parse_time(t)

Given a input as ticks or string try to return its datetime equivalent.

resolution()

round_ticks(inticks, interval)

Tick-based version of round_time in seconds, for use with vectors ticks : ticks representing seconds interval: interval to round to (non-calendar dependent)

round_time(dtime, interval)

Round a datetime object to an interval dtime : datetime.datetime object, default now. interval: interval to round to (non-calendar dependent) returns rounded time and offset in seconds between input time and rounded time

seconds(s)

Create a time interval representing s seconds

ticks(time, base=datetime.datetime(1, 1, 1, 0, 0))

Convert a datetime to a long integer. The long integer is a number of units since a time base. The actual value depends on resolution.

ticks_to_interval(ticks, base=datetime.datetime(1, 1, 1, 0, 0))

Convert a number of ticks to a time interval.

ticks_to_time(ticks, base=datetime.datetime(1, 1, 1, 0, 0))

Convert a number of ticks to a datetime

time_interval(years=0, months=0, days=0, hours=0, minutes=0, seconds=0)

Create a time interval. Create a time interval representing the inputs that is appropriate for VTools. This function is the ‘hard way’ to create a time interval. However, it is useful when the interval is not known in advanced and must be created programatically.

time_sequence(start, dt, n)

Create ticks sequence based on provided start datetime, and timedelta/relativedelta dt start must be instance of datetime. input:

start: must be a instance of datetime, dt: maybe timedelta or relativedelta. n: number of time points, not number of intervals.

return:

A time sequence array in ticks.

years(y)

Create a time interval representing y years

Module contents