Accessing Data in a Database 


Tutorial 110 connects the strategy to a database table containing historical S&P-500 price data. In this tutorial we are attaching the strategy to an object of type fast::database. fast::database is a class allowing for very fast access to time series data - millions of records per second. Data can also be accessed from an SQLite database as well as from PostgreSQL. More database drivers will likely be available at some future point. Please refer to chapter 5 for more information on databases. The database access method is identical for all databases. Just replace class fast::database with class postgres::database to read data from a postgresql based series table. The strategy class itself is not aware of the source of the data.

 

 

 

 

 

 

 

 

 

 

(1)

 

 

 

 

 

(2)

 

 

 

 

 

 

 

(3)

 

(4)

 

 

(5)

 

(6)

 

#include "tsa.h"

using namespace tsa;

 

class my_strategy : public strategy

{

          in_stream sp500;

 

          void on_start(void)   override

          {

                    sp500.connect("sp500.daily");

          }

 

          void on_bar_close(void)   override

          {

                    std::cout

                              << "timestamp: " << (date)sp500.timestamp() << std::endl 

                              << "open:      " << sp500.open[0]     << std::endl

                              << "close:     " << sp500.close[0]    << std::endl;

          }

};

 

 

          fast::database db("output/demo_db", create_db | truncate_db);

 

          import_from_file__SP500_daily(db);

 

          my_strategy s;

          s.attach(db);

                 

          s.run("2010-01-01","2010-01-10");

     //s.run();

 

Program Output

 

Tutorial:    110

=================================

importing data...

timestamp: 2009-12-31

open:      1123.5

close:     1110.7

timestamp: 2010-01-04

open:      1119.5

close:     1128.8

timestamp: 2010-01-05

open:      1128.3

close:     1132.3

(...lines skipped...)

press any key to exit...

 

 

 

(1)

As part of the strategy's on_start() definition we have the following line:

 

       sp500.connect("sp500.daily");

 

The stream's connect() member connects it to the given data source, in this case an existing database table with the name 'sp500.daily'. ( In other words, the 'connect' member 'opens' a stream. The reason why the member is called 'connect' and not 'open' is because an 'open' member already exists representing the 'open price' series.)

 

How does the stream object know which database to connect to?

 

A few lines below, starting at (3) we instantiate a database object called 'db'. We then import some data from a file, and eventually attach this database object to the strategy in (5) using strategy::attach().

 

(3)          fast::database db("output/demo_db", create_db | truncate_db);

 

          my_strategy s;

(5)          s.attach(db);

 

So when a stream wants to open a database table, it first checks which data-source the strategy is attached to, and then uses this data-source to connect to a specific table or stream if it exists.

 

It is possible to attach multiple database objects to a single strategy but in such cases each must be given a unique name using strategy::attach_as(). This will be discussed in later chapters.

(2)

std::cout

          << "timestamp: " << (date)sp500.timestamp() << std::endl 

          << "open:      " << sp500.open[0]     << std::endl

          << "close:     " << sp500.close[0]    << std::endl;

 

This line prints the timestamp as well as the current open and closing prices to standard output. Recall that the 'sp500' in_stream object automatically organizes incoming records into series, of type series<T>. Individual values are accessed using the subscript operator[]. You will learn more of class series<T> shortly.

(4)

 

import_from_file__SP500_daily(db);

 

This is a helper function defined in file tutorial__data_import.cpp and simply imports a text file with historical records for the S&P 500 stock index. The file can be found in the 'tutorial/data/' sub-directory.

 

void import_from_file__SP500_daily(fast::database& db)

{

          file_import_rules p;

 

          p.file_path = "data/sp500_cont.csv";

          p.target_table = "sp500.daily";

          p.date_format = "mm/dd/yyyy";

          p.columns = "open,high,low,close,volume,openint";

          p.field_separator = ',';

          p.delete_existing_table = true;

 

          db.import(p);

}

 

 

 You may have noticed that this program's output started with:

  timestamp: 2009-12-31

 

even though the strategy start was set for 2010-01-01.  This discrepancy occurs because when the strategy requests data for a particular date, and the underlying data source does not have a record for the given date, then it returns the next best value, in other words the most recently available value for a given time-point. Since markets didn't trade on Jan 1 of 2010 due to the new year holiday, the data returned was for the most recent trading day - which was Dec-31-2009.

 

Since the strategy is normally scheduled by a time-point sequence coming directly from the data source or data feed itself, such a discrepancy is not possible. In this particular case, the start time-point was user mandated, and did not originate from the data source.

 

This type of fall-back behaviour can also occur when a strategy receives data from multiple unsynchronized data streams, like streams from markets with different holiday gaps.