Aggregating and combining intraday data

Importing and Managing Financial Data in R

Joshua Ulrich

Quantitative Analyst & quantmod Co-Author and Maintainer

Timezones!

¹ Source: https://www.shutterstock.com/video/search/time-zone-clocks

Timezones!

Internally, xts index is seconds since midnight 1970-01-01 in UTC
merge() uses internal index
merge() result will have timezone of the first object

Timezones!

datetime <- as.POSIXct("2017-01-18 10:00:00", tz = "UTC")
london <- xts(1, datetime, tzone = "Europe/London")
tokyo <- xts(1, datetime, tzone = "Asia/Tokyo")

merge(london, tokyo)

                    london tokyo
2017-01-18 10:00:00      1     1

merge(tokyo, london)

                    tokyo london
2017-01-18 19:00:00     1      1

Creating regular intraday data

head(dc_trades)

                    Price
2016-01-16 08:00:58 20.85
2016-01-16 08:01:56 20.85
2016-01-16 08:03:35 20.85
2016-01-16 08:07:44 20.84
2016-01-16 08:45:58 20.85
2016-01-16 08:46:49 20.85

Creating regular intraday data

datetimes <- seq(from = as.POSIXct("2016-01-16 08:00"),
                  to = as.POSIXct("2016-01-17 18:00"),
                  by = "1 min")

regular_xts <- xts(, order.by = datetimes)

merged_xts <- merge(dc_trades, regular_xts)
head(merged_xts)

                    Price
2016-01-16 08:00:00    NA
2016-01-16 08:00:58 20.85
2016-01-16 08:01:00    NA
2016-01-16 08:01:56 20.85
2016-01-16 08:02:00    NA
2016-01-16 08:03:00    NA

Subset to trading hours

# All observations should be NA
all(is.na(merged_xts["2016-01-16 19:00/2016-01-17 07:00"]))

TRUE

# xts time-of-day subsetting
merged_trade_day <- merged_xts["T08:00/T18:00"]

# Now there are no observations
nrow(merged_trade_day["2016-01-16 19:00/2016-01-17 07:00"])

Fill missing values by trading day

split()-lapply()-rbind() paradigm from this DataCamp course about manipulating time series

# split() data into list of non-overlapping chunks
trade_day_list <- split(merged_trade_day, "days")

# lapply() a function to each chunk (list element)
filled_trade_day_list <- lapply(trade_day_list, na.locf)

# Combine list of chunks using do.call() and rbind()
filled_trade_day <- do.call(rbind, filled_trade_day_list)

Aggregate irregular intraday data

Aggregate dense intraday data with to.period()
- period: new periodicity (e.g. seconds, hours, days, etc)
- k: number of periods per new observation

Aggregate irregular intraday data (1)

head(dc_price)

                    DC.Price
2016-01-16 00:00:07 20.84224
2016-01-16 00:00:08 20.84225
2016-01-16 00:00:08 20.84225
2016-01-16 00:00:11 20.84225
2016-01-16 00:00:25 20.84224
2016-01-16 00:00:44 20.84224

xts_5min <- to.period(dc_price, period = "minutes", k = 5)
head(xts_5min, n = 4)

                    dc_price.Open  dc_price.High  dc_price.Low  dc_price.Close
2016-01-16 00:03:49      20.84224       20.84227      20.84140        20.84160
2016-01-16 00:09:50      20.84160       20.84160      20.84156        20.84156
2016-01-16 00:14:57      20.84156       20.84156      20.84154        20.84154
2016-01-16 00:19:23      20.84154       20.84154      20.83206        20.83211

xts_aligned <- align.time(xts_5min, n = 60 * 5)
head(xts_aligned, n = 4)

                    dc_price.Open  dc_price.High  dc_price.Low  dc_price.Close
2016-01-16 00:05:00      20.84224       20.84227      20.84140        20.84160
2016-01-16 00:05:00      20.84160       20.84160      20.84156        20.84156
2016-01-16 00:15:00      20.84156       20.84156      20.84154        20.84154
2016-01-16 00:20:00      20.84154       20.84154      20.83206        20.83211

Let's practice!

Importing and Managing Financial Data in R