Questions tagged [time-series]
A Time series is a sequence of data points with values measured at successive times (either in continuous time or at discrete time periods). Time series analysis exploits this natural temporal ordering to extract meaning and trends from the underlying data.
13,500
questions
351votes
35answers
257kviews
Peak signal detection in realtime timeseries data
Update: The best performing algorithm so far is this one.
This question explores robust algorithms for detecting sudden peaks in real-time timeseries data.
Consider the following example data:
...
- 17.7k
341votes
5answers
562kviews
Plotting two variables as lines using ggplot2 on the same graph
A very newbish question, but say I have data like this:
test_data <-
data.frame(
var0 = 100 + c(0, cumsum(runif(49, -20, 20))),
var1 = 150 + c(0, cumsum(runif(49, -10, 10))),
date = ...
- 53.6k
191votes
10answers
68kviews
Storing time-series data, relational or non?
I am creating a system which polls devices for data on varying metrics such as CPU utilisation, disk utilisation, temperature etc. at (probably) 5 minute intervals using SNMP. The ultimate goal is to ...
- 18.9k
184votes
18answers
307kviews
How to calculate rolling / moving average using python + NumPy / SciPy?
There seems to be no function that simply calculates the moving average on numpy/scipy, leading to convoluted solutions.
My question is two-fold:
What's the easiest way to (correctly) implement a ...
- 20k
167votes
13answers
168kviews
Combine Date and Time columns using python pandas
I have a pandas dataframe with the following columns:
data = {'Date': ['01-06-2013', '02-06-2013', '02-06-2013', '02-06-2013', '02-06-2013', '03-06-2013', '03-06-2013', '03-06-2013', '03-06-2013', '04-...
- 14.8k
127votes
10answers
125kviews
Can Pandas plot a histogram of dates?
I've taken my Series and coerced it to a datetime column of dtype=datetime64[ns] (though only need day resolution...not sure how to change).
import pandas as pd
df = pd.read_csv('somefile.csv')
...
- 14.4k
126votes
4answers
88kviews
How to get a vertical geom_vline to an x-axis of class date?
Even though I found Hadley's post in the google group on POSIXct and geom_vline, I could not get it done. I have a time series from and would like to draw a vertical line for years 1998, 2005 and 2010 ...
- 26.3k
116votes
4answers
135kviews
Generating time series between two dates in PostgreSQL
I have a query like this that nicely generates a series of dates between 2 given dates:
select date '2004-03-07' + j - i as AllDate
from generate_series(0, extract(doy from date '2004-03-07')::int - ...
- 5,169
105votes
9answers
158kviews
Pandas: rolling mean by time interval
I've got a bunch of polling data; I want to compute a Pandas rolling mean to get an estimate for each day based on a three-day window. According to this question, the rolling_* functions compute the ...
- 2,074
88votes
2answers
50kviews
How to parse milliseconds?
How do I use strptime or any other functions to parse time stamps with milliseconds in R?
time[1]
# [1] "2010-01-15 13:55:23.975"
strptime(time[1], format="%Y-%m-%d %H:%M:%S.%f")
# [1] NA
strptime(...
- 3,990
83votes
9answers
85kviews
auto.arima() equivalent for python
I am trying to predict weekly sales using ARMA ARIMA models. I could not find a function for tuning the order(p,d,q) in statsmodels. Currently R has a function forecast::auto.arima() which will tune ...
- 1,579
74votes
8answers
40kviews
Resampling Within a Pandas MultiIndex
I have some hierarchical data which bottoms out into time series data which looks something like this:
df = pandas.DataFrame(
{'value_a': values_a, 'value_b': values_b},
index=[states, cities,...
- 850
74votes
5answers
55kviews
Pattern recognition in time series [closed]
By processing a time series graph, I Would like to detect patterns that look similar to this:
Using a sample time series as an example, I would like to be able to detect the patterns as marked here:
...
- 937
63votes
6answers
38kviews
Is there a powerful database system for time series data? [closed]
In multiple projects we have to store, aggregate, evaluate simple measurement values. One row typcially consists of a time stamp, a value and some attributes to the value. In some applications we ...
- 4,220
61votes
5answers
55kviews
Pandas: resample timeseries with groupby
Given the below pandas DataFrame:
In [115]: times = pd.to_datetime(pd.Series(['2014-08-25 21:00:00','2014-08-25 21:04:00',
'2014-08-25 22:07:00','2014-08-...
- 635